linux-rt-users.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Steven Rostedt <rostedt@goodmis.org>
To: linux-kernel@vger.kernel.org,
	linux-rt-users <linux-rt-users@vger.kernel.org>
Cc: Thomas Gleixner <tglx@linutronix.de>,
	Carsten Emde <C.Emde@osadl.org>,
	Sebastian Andrzej Siewior <bigeasy@linutronix.de>,
	John Kacur <jkacur@redhat.com>, <stable-rt@vger.kernel.org>,
	Steven Rostedt <srostedt@redhat.com>
Subject: [PATCH RT 14/19] hwlat-detector: Update hwlat_detector to add outer loop detection
Date: Sun, 08 Sep 2013 11:14:09 -0400	[thread overview]
Message-ID: <20130908151448.202985066@goodmis.org> (raw)
In-Reply-To: 20130908151355.362583092@goodmis.org

[-- Attachment #1: 0014-hwlat-detector-Update-hwlat_detector-to-add-outer-lo.patch --]
[-- Type: text/plain, Size: 3993 bytes --]

From: Steven Rostedt <rostedt@goodmis.org>

The hwlat_detector reads two timestamps in a row, then reports any
gap between those calls. The problem is, it misses everything between
the second reading of the time stamp to the first reading of the time stamp
in the next loop. That's were most of the time is spent, which means,
chances are likely that it will miss all hardware latencies. This
defeats the purpose.

By also testing the first time stamp from the previous loop second
time stamp (the outer loop), we are more likely to find a latency.

Setting the threshold to 1, here's what the report now looks like:

1347415723.0232202770	0	2
1347415725.0234202822	0	2
1347415727.0236202875	0	2
1347415729.0238202928	0	2
1347415731.0240202980	0	2
1347415734.0243203061	0	2
1347415736.0245203113	0	2
1347415738.0247203166	2	0
1347415740.0249203219	0	3
1347415742.0251203272	0	3
1347415743.0252203299	0	3
1347415745.0254203351	0	2
1347415747.0256203404	0	2
1347415749.0258203457	0	2
1347415751.0260203510	0	2
1347415754.0263203589	0	2
1347415756.0265203642	0	2
1347415758.0267203695	0	2
1347415760.0269203748	0	2
1347415762.0271203801	0	2
1347415764.0273203853	2	0

There's some hardware latency that takes 2 microseconds to run.

Cc: stable-rt@vger.kernel.org
Signed-off-by: Steven Rostedt <srostedt@redhat.com>
Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
---
 drivers/misc/hwlat_detector.c |   32 ++++++++++++++++++++++++++------
 1 file changed, 26 insertions(+), 6 deletions(-)

diff --git a/drivers/misc/hwlat_detector.c b/drivers/misc/hwlat_detector.c
index b7b7c90..f93b8ef 100644
--- a/drivers/misc/hwlat_detector.c
+++ b/drivers/misc/hwlat_detector.c
@@ -143,6 +143,7 @@ static void detector_exit(void);
 struct sample {
 	u64		seqnum;		/* unique sequence */
 	u64		duration;	/* ktime delta */
+	u64		outer_duration;	/* ktime delta (outer loop) */
 	struct timespec	timestamp;	/* wall time */
 	unsigned long   lost;
 };
@@ -219,11 +220,13 @@ static struct sample *buffer_get_sample(struct sample *sample)
  */
 static int get_sample(void *unused)
 {
-	ktime_t start, t1, t2;
+	ktime_t start, t1, t2, last_t2;
 	s64 diff, total = 0;
 	u64 sample = 0;
+	u64 outer_sample = 0;
 	int ret = 1;
 
+	last_t2.tv64 = 0;
 	start = ktime_get(); /* start timestamp */
 
 	do {
@@ -231,7 +234,22 @@ static int get_sample(void *unused)
 		t1 = ktime_get();	/* we'll look for a discontinuity */
 		t2 = ktime_get();
 
+		if (last_t2.tv64) {
+			/* Check the delta from the outer loop (t2 to next t1) */
+			diff = ktime_to_us(ktime_sub(t1, last_t2));
+			/* This shouldn't happen */
+			if (diff < 0) {
+				printk(KERN_ERR BANNER "time running backwards\n");
+				goto out;
+			}
+			if (diff > outer_sample)
+				outer_sample = diff;
+		}
+		last_t2 = t2;
+
 		total = ktime_to_us(ktime_sub(t2, start)); /* sample width */
+
+		/* This checks the inner loop (t1 to t2) */
 		diff = ktime_to_us(ktime_sub(t2, t1));     /* current diff */
 
 		/* This shouldn't happen */
@@ -246,12 +264,13 @@ static int get_sample(void *unused)
 	} while (total <= data.sample_width);
 
 	/* If we exceed the threshold value, we have found a hardware latency */
-	if (sample > data.threshold) {
+	if (sample > data.threshold || outer_sample > data.threshold) {
 		struct sample s;
 
 		data.count++;
 		s.seqnum = data.count;
 		s.duration = sample;
+		s.outer_duration = outer_sample;
 		s.timestamp = CURRENT_TIME;
 		__buffer_add_sample(&s);
 
@@ -738,10 +757,11 @@ static ssize_t debug_sample_fread(struct file *filp, char __user *ubuf,
 		}
 	}
 
-	len = snprintf(buf, sizeof(buf), "%010lu.%010lu\t%llu\n",
-		      sample->timestamp.tv_sec,
-		      sample->timestamp.tv_nsec,
-		      sample->duration);
+	len = snprintf(buf, sizeof(buf), "%010lu.%010lu\t%llu\t%llu\n",
+		       sample->timestamp.tv_sec,
+		       sample->timestamp.tv_nsec,
+		       sample->duration,
+		       sample->outer_duration);
 
 
 	/* handling partial reads is more trouble than it's worth */
-- 
1.7.10.4

  parent reply	other threads:[~2013-09-08 15:14 UTC|newest]

Thread overview: 20+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2013-09-08 15:13 [PATCH RT 00/19] 3.6.11.8-rt41-rc1 stable review Steven Rostedt
2013-09-08 15:13 ` [PATCH RT 01/19] sched/workqueue: Only wake up idle workers if not blocked on sleeping spin lock Steven Rostedt
2013-09-08 15:13 ` [PATCH RT 02/19] x86/mce: fix mce timer interval Steven Rostedt
2013-09-08 15:13 ` [PATCH RT 03/19] genirq: Set irq thread to RT priority on creation Steven Rostedt
2013-09-08 15:13 ` [PATCH RT 04/19] list_bl.h: make list head locking RT safe Steven Rostedt
2013-09-08 15:14 ` [PATCH RT 05/19] list_bl.h: fix it for for !SMP && !DEBUG_SPINLOCK Steven Rostedt
2013-09-08 15:14 ` [PATCH RT 06/19] timers: prepare for full preemption improve Steven Rostedt
2013-09-08 15:14 ` [PATCH RT 07/19] kernel/cpu: fix cpu down problem if kthreads cpu is going down Steven Rostedt
2013-09-08 15:14 ` [PATCH RT 08/19] kernel/hotplug: restore original cpu mask oncpu/down Steven Rostedt
2013-09-08 15:14 ` [PATCH RT 09/19] gpu: i915: allow the user not to do the wbinvd Steven Rostedt
2013-09-08 15:14 ` [PATCH RT 10/19] drm/i915: drop trace_i915_gem_ring_dispatch on rt Steven Rostedt
2013-09-08 15:14 ` [PATCH RT 11/19] sched: Distangle worker accounting from rqlock Steven Rostedt
2013-09-08 15:14 ` [PATCH RT 12/19] rt,ntp: Move call to schedule_delayed_work() to helper thread Steven Rostedt
2013-09-08 15:14 ` [PATCH RT 13/19] hpsa: fix warning with smp_processor_id() in preemptible section Steven Rostedt
2013-09-08 15:14 ` Steven Rostedt [this message]
2013-09-08 15:14 ` [PATCH RT 15/19] hwlat-detect/trace: Export trace_clock_local for hwlat-detector Steven Rostedt
2013-09-08 15:14 ` [PATCH RT 16/19] hwlat-detector: Use trace_clock_local if available Steven Rostedt
2013-09-08 15:14 ` [PATCH RT 17/19] hwlat-detector: Use thread instead of stop machine Steven Rostedt
2013-09-08 15:14 ` [PATCH RT 18/19] genirq: do not invoke the affinity callback via a workqueue Steven Rostedt
2013-09-08 15:14 ` [PATCH RT 19/19] Linux 3.6.11.8-rt41-rc1 Steven Rostedt

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20130908151448.202985066@goodmis.org \
    --to=rostedt@goodmis.org \
    --cc=C.Emde@osadl.org \
    --cc=bigeasy@linutronix.de \
    --cc=jkacur@redhat.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-rt-users@vger.kernel.org \
    --cc=srostedt@redhat.com \
    --cc=stable-rt@vger.kernel.org \
    --cc=tglx@linutronix.de \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).