Re: Measuring scheduling latency for RT threads

[Clark Williams <williams@redhat.com>]

[-- Attachment #1: Type: text/plain, Size: 1581 bytes --]

On Wed, 19 Nov 2014 14:19:49 +0100
Stanislav Meduna <stano@meduna.org> wrote:

> On 19.11.2014 13:43, "Jürgen Lanner" wrote:
> 
> > My first goal is to find out about the worst case latency:
> > Is there a way I can find out how long (worst case) a RT thread
> > being ready to run is just waiting to be dispatched?  
> 
> ftrace, trace-cmd, kernelshark
> 
> The latency for the highest prio runnable task is available
> right away with the wakeup tracer. For other tasks you can
> trace the scheduling functions and interpret the results
> using a script (or look at them graphically using kernelshark).
> 
> Note that the function tracing is not free and will skew
> the results a bit, but should be good enough to identify
> the offenders.
> 

You might want to establish a baseline of what the expected latency on
your hardware will be. Try the 'cyclictest' application from the
rt-tests package. Run that for some number of hours to see if you have
any hardware issues that may cause delays. 

When run this way:

	$ sudo cyclictest --numa -p95 -mu

The program will kick off a measurement thread at fifo:95 on each
online core. It will the loop until you kill it with ^c, sleeping for
an interval and then measuring the delta between ideal wakeup time and
actual wakeup time of the measurement thread, something like this:

	t1 = gettime()
	loop:
		sleep(interval)
		t2=gettime()
		delta = t2 - (t1+interval)
		print delta
		t1=gettime()
		goto loop

That will show you the actual scheduling latency seen on each core. 

Clark

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]