[RFC v0 0/9] [RFC v0 0/9] pi_stress: Add SCHED_DEADLINE support

[RFC v0 0/9] [RFC v0 0/9] pi_stress: Add SCHED_DEADLINE support

[Daniel Wagner]

Hi,

I wrote a small test program to play around with SCHED_DEADLINE which
triggered various PI related crashes. Steven convinced me to get
the test somehow upstream. I figured pi_stress is very simular with my
test program. So here my attempt to merge the essence of my test
into pi_stress.c


  patch 1: hackbench: Don't re-assign context for each fd
  patch 2: rt-tests.h: Remove unused header file
  patch 3: pi_stress: Remove unused TIMER_SIGNAL definition

	Not really related to this series. Just found it while
	browsing the code a bit. Are you interested in such
	patches?

  patch 4: error: Add debug() function
  patch 5: pi_stress: Use error.h for logging and debugging

	I didn't want to add the sched_setattr() code directly
	to pi_stress.c. As soon rt-utils.h is used you get also
	error.h which brings info() and friends it collides
	in pi_stress.c.
	Several other tests also implement info() and friends.
	Would it make sense to convert them using error.h too?
	Then I would also suggest to get --verbose and
	--debug options handled in a central way. Maybe something
	like the GLib does with parsing the argc first and hanlding
	those flags. What do you think?

  patch 6: rt-sched: Add sched_setattr/sched_getattr API
  patch 7: rt-utils: Add helper to parse/print scheduling policies
  patch 8: rt-utils: Add gettid()

	Add to the library some of the code which could be reused
	by other tests as well. Nothing really fancy.

  patch 9: pi_stress: Store schedule attributes per thread

	Finally this adds the a command line arguments which
	allows the user to define which scheduler setting is
	used for each thread catagory:

# ./pi_stress --sched id=high,policy=deadline,runtime=100000,deadline=200000,period=200000
Starting PI Stress Test
Number of thread groups: 1
Duration of test run: infinite
Number of inversions per group: unlimited
     Admin thread SCHED_FIFO priority 4
1 groups of 3 threads will be created
      High thread SCHED_DEADLINE runtime 100000 deadline 200000 period 200000
       Med thread SCHED_FIFO priority 2
       Low thread SCHED_FIFO priority 1
Current Inversions: 2446249
Stopping test
Terminated


WARNING: Do no run this on older kernels (<3.18-rc3) it will crash your
system. You have been warned!


Obviously, you can also do some stupid stuff like:

# ./pi_stress --rr --sched id=high,policy=deadline,h=dl,runtime=100000,deadline=200000,period=200000 --sched id=low,policy=fifo,priority=3
Starting PI Stress Test
Number of thread groups: 1
Duration of test run: infinite
Number of inversions per group: unlimited
     Admin thread SCHED_RR priority 4
1 groups of 3 threads will be created
      High thread SCHED_DEADLINE runtime 100000 deadline 200000 period 200000
       Med thread SCHED_RR priority 2
       Low thread SCHED_FIFO priority 3
Current Inversions: 352306
Stopping test
Terminated

cheers,
daniel

Cc: Clark Williams <williams@redhat.com>
Cc: John Kacur <jkacur@redhat.com>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Dario Faggioli <raistlin@linux.it>
Cc: <linux-rt-users@vger.kernel.org>

Daniel Wagner (9):
  hackbench: Don't re-assign context for each fd
  rt-tests.h: Remove unused header file
  pi_stress: Remove unused TIMER_SIGNAL definition
  error: Add debug() function
  pi_stress: Use error.h for logging and debugging
  rt-sched: Add sched_setattr/sched_getattr API
  rt-utils: Add helper to parse/print scheduling policies
  rt-utils: Add gettid()
  pi_stress: Store schedule attributes per thread

 Makefile                  |   4 +-
 src/hackbench/hackbench.c |   8 +-
 src/include/error.h       |   1 +
 src/include/rt-sched.h    |  76 +++++++
 src/include/rt-tests.h    | 124 ------------
 src/include/rt-utils.h    |   7 +
 src/lib/error.c           |  10 +
 src/lib/rt-sched.c        |  43 ++++
 src/lib/rt-utils.c        |  44 ++++
 src/pi_tests/pi_stress.c  | 500 ++++++++++++++++++++++++++++++----------------
 10 files changed, 512 insertions(+), 305 deletions(-)
 create mode 100644 src/include/rt-sched.h
 delete mode 100644 src/include/rt-tests.h
 create mode 100644 src/lib/rt-sched.c

-- 
1.9.3

[RFC v0 1/9] hackbench: Don't re-assign context for each fd

[Daniel Wagner]

A small optimization. Setting it once is enough.

Signed-off-by: Daniel Wagner <daniel.wagner@bmw-carit.de>
---
 src/hackbench/hackbench.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/src/hackbench/hackbench.c b/src/hackbench/hackbench.c
index 34a27ef..c42257c 100644
--- a/src/hackbench/hackbench.c
+++ b/src/hackbench/hackbench.c
@@ -326,12 +326,12 @@ static unsigned int group(childinfo_t *child,
 			close(fds[0]);
 	}
 
+	snd_ctx->ready_out = ready_out;
+	snd_ctx->wakefd = wakefd;
+	snd_ctx->num_fds = num_fds;
+
 	/* Now we have all the fds, fork the senders */
 	for (i = 0; i < num_fds; i++) {
-		snd_ctx->ready_out = ready_out;
-		snd_ctx->wakefd = wakefd;
-		snd_ctx->num_fds = num_fds;

[RFC v0 2/9] rt-tests.h: Remove unused header file

[Daniel Wagner]

There is no user of this header file.

Signed-off-by: Daniel Wagner <daniel.wagner@bmw-carit.de>
---
 src/include/rt-tests.h | 124 -------------------------------------------------
 1 file changed, 124 deletions(-)
 delete mode 100644 src/include/rt-tests.h

diff --git a/src/include/rt-tests.h b/src/include/rt-tests.h
deleted file mode 100644
index d7eefd9..0000000
--- a/src/include/rt-tests.h
+++ /dev/null
@@ -1,124 +0,0 @@
-#ifndef __RT_TESTS_H__
-#define __RT_TESTS_H__
-
-#include <stdio.h>
-#include <stdlib.h>
-#include <unistd.h>
-#define __USE_GNU
-#include <fcntl.h>
-#include <getopt.h>
-#include <pthread.h>
-#include <signal.h>
-#include <sched.h>
-#include <string.h>
-#include <time.h>
-#include <errno.h>
-
-#include <linux/unistd.h>
-
-#include <sys/prctl.h>
-#include <sys/stat.h>
-#include <sys/sysinfo.h>
-#include <sys/types.h>
-#include <sys/time.h>
-#include <sys/utsname.h>
-#include <sys/mman.h>
-
-#ifndef SCHED_IDLE
-#define SCHED_IDLE 5
-#endif
-#ifndef SCHED_NORMAL
-#define SCHED_NORMAL SCHED_OTHER
-#endif
-
-/* Ugly, but .... */
-#define gettid() syscall(__NR_gettid)
-#define sigev_notify_thread_id _sigev_un._tid
-
-#ifdef __UCLIBC__
-#define MAKE_PROCESS_CPUCLOCK(pid, clock) \
-	((~(clockid_t) (pid) << 3) | (clockid_t) (clock))
-#define CPUCLOCK_SCHED          2
-
-static int clock_nanosleep(clockid_t clock_id, int flags, const struct timespec *req,
-		struct timespec *rem)
-{
-	if (clock_id == CLOCK_THREAD_CPUTIME_ID)
-		return -EINVAL;
-	if (clock_id == CLOCK_PROCESS_CPUTIME_ID)
-		clock_id = MAKE_PROCESS_CPUCLOCK (0, CPUCLOCK_SCHED);
-
-	return syscall(__NR_clock_nanosleep, clock_id, flags, req, rem);
-}
-
-static int sched_setaffinity(pid_t pid, unsigned int cpusetsize,
-		cpu_set_t *mask)
-{
-	return -EINVAL;
-}
-
-static void CPU_SET(int cpu, cpu_set_t *set) { }
-static void CPU_ZERO(cpu_set_t *set) { }
-#else
-extern int clock_nanosleep(clockid_t __clock_id, int __flags,
-			   __const struct timespec *__req,
-			   struct timespec *__rem);
-#endif
-
-#define USEC_PER_SEC		1000000
-#define NSEC_PER_SEC		1000000000
-
-#define HIST_MAX		1000000
-
-/* Struct to transfer parameters to the thread */
-struct thread_param {
-	int prio;
-	int policy;
-	int mode;
-	int timermode;
-	int signal;
-	int clock;
-	unsigned long max_cycles;
-	struct thread_stat *stats;
-	int bufmsk;
-	unsigned long interval;
-	int cpu;
-};
-
-/* Struct for statistics */
-struct thread_stat {
-	unsigned long cycles;
-	unsigned long cyclesread;
-	long min;
-	long max;
-	long act;
-	double avg;
-	long *values;
-	long *hist_array;
-	pthread_t thread;
-	int threadstarted;
-	int tid;
-	long reduce;
-	long redmax;
-	long cycleofmax;
-	long hist_overflow;
-};
-
-enum kernelversion {
-	KV_NOT_26,	/* not a 2.6 kernel */
-	KV_26_LT18,	/* less than 2.6.18 */
-	KV_26_LT24,	/* less than 2.6.24 */
-	KV_26_LT28,	/* less than 2.6.28 */
-	KV_26_CURR,	/* 2.6.28+          */
-};
-
-enum kernelversion check_kernel(void);
-
-enum {
-	ERROR_GENERAL	= -1,
-	ERROR_NOTFOUND	= -2,
-};
-
-#define TIMER_RELTIME		0

[RFC v0 3/9] pi_stress: Remove unused TIMER_SIGNAL definition

[Daniel Wagner]

Signed-off-by: Daniel Wagner <daniel.wagner@bmw-carit.de>
---
 src/pi_tests/pi_stress.c | 2 --
 1 file changed, 2 deletions(-)

diff --git a/src/pi_tests/pi_stress.c b/src/pi_tests/pi_stress.c
index e273d62..39e6a37 100644
--- a/src/pi_tests/pi_stress.c
+++ b/src/pi_tests/pi_stress.c
@@ -157,8 +157,6 @@ int prio_min;
 #define NUM_TEST_THREADS 3
 #define NUM_ADMIN_THREADS 1
 
-#define TIMER_SIGNAL	(SIGRTMIN+1)

[RFC v0 4/9] error: Add debug() function

[Daniel Wagner]

Signed-off-by: Daniel Wagner <daniel.wagner@bmw-carit.de>
---
 src/include/error.h |  1 +
 src/lib/error.c     | 10 ++++++++++
 2 files changed, 11 insertions(+)

diff --git a/src/include/error.h b/src/include/error.h
index 1e33f6c..ae05a2e 100644
--- a/src/include/error.h
+++ b/src/include/error.h
@@ -10,6 +10,7 @@ void err_exit(int err, char *fmt, ...);
 void err_msg(char *fmt, ...);
 void err_msg_n(int err, char *fmt, ...);
 void err_quit(char *fmt, ...);
+void debug(char *fmt, ...);
 void info(char *fmt, ...);
 void warn(char *fmt, ...);
 void fatal(char *fmt, ...);
diff --git a/src/lib/error.c b/src/lib/error.c
index 1b5de5d..0cef245 100644
--- a/src/lib/error.c
+++ b/src/lib/error.c
@@ -46,6 +46,16 @@ void err_quit(char *fmt, ...)
 	exit(1);
 }
 
+void debug(char *fmt, ...)
+{
+	va_list ap;
+
+	va_start(ap, fmt);
+	fputs("DEBUG: ", stderr);
+	err_doit(0, fmt, ap);
+	va_end(ap);
+}
+
 void info(char *fmt, ...)
 {
 	va_list ap;
-- 
1.9.3

[RFC v0 5/9] pi_stress: Use error.h for logging and debugging

[Daniel Wagner]

In order to be able to use some of the rt-utils.h function we need
to get rid of our own info() & friends implementation.

Signed-off-by: Daniel Wagner <daniel.wagner@bmw-carit.de>
---
 src/pi_tests/pi_stress.c | 255 +++++++++++++++++++++--------------------------
 1 file changed, 115 insertions(+), 140 deletions(-)

diff --git a/src/pi_tests/pi_stress.c b/src/pi_tests/pi_stress.c
index 39e6a37..d834b86 100644
--- a/src/pi_tests/pi_stress.c
+++ b/src/pi_tests/pi_stress.c
@@ -55,6 +55,8 @@
 #include <sys/wait.h>
 #include <termios.h>
 
+#include "error.h"
+
 /* conversions */
 #define USEC_PER_SEC 	1000000
 #define NSEC_PER_SEC 	1000000000
@@ -85,6 +87,13 @@
 #define UP_ONE "\033[1A"
 #define DOWN_ONE "\033[1B"
 
+#define pi_info(fmt, arg...) \
+	do { if (verbose) info(fmt, ## arg); } while (0)
+#define pi_debug(fmt, arg...) \
+	do { if (debugging) debug(fmt, ## arg); } while (0)
+#define pi_error(fmt, arg...) \
+	do { err_msg(fmt, ## arg); have_errors = 1; } while (0)
+
 /* the length of the test */
 /* default is infinite */
 int duration = -1;
@@ -102,7 +111,7 @@ int inversions = -1;
 /* turn on lots of prints */
 int verbose = 0;
 
-/* turn on debugging prints */
+/* turn on pi_debugging prints */
 int debugging = 0;
 
 int quiet = 0;	/* turn off all prints, default = 0 (off) */
@@ -220,9 +229,6 @@ void *watchdog(void *arg);
 int setup_thread_attr(pthread_attr_t * attr, int prio, cpu_set_t * mask,
 		      int schedpolicy);
 int set_cpu_affinity(cpu_set_t * test_mask, cpu_set_t * admin_mask);
-void error(char *, ...);
-void info(char *, ...);
-void debug(char *, ...);
 void process_command_line(int argc, char **argv);
 void usage(void);
 int block_signals(void);
@@ -261,7 +267,7 @@ int main(int argc, char **argv)
 	/* lock memory */
 	if (lockall)
 		if (mlockall(MCL_CURRENT | MCL_FUTURE) == -1) {
-			error("mlockall failed\n");
+			pi_error("mlockall failed\n");
 			return FAILURE;
 		}
 	/* boost main's priority (so we keep running) :) */
@@ -269,7 +275,7 @@ int main(int argc, char **argv)
 	thread_param.sched_priority = MAIN_PRIO();
 	status = pthread_setschedparam(pthread_self(), policy, &thread_param);
 	if (status) {
-		error("main: boosting to max priority: 0x%x\n", status);
+		pi_error("main: boosting to max priority: 0x%x\n", status);
 		return FAILURE;
 	}
 	/* block unwanted signals */
@@ -278,7 +284,7 @@ int main(int argc, char **argv)
 	/* allocate our groups array */
 	groups = calloc(ngroups, sizeof(struct group_parameters));
 	if (groups == NULL) {
-		error("main: failed to allocate %d groups\n", ngroups);
+		pi_error("main: failed to allocate %d groups\n", ngroups);
 		return FAILURE;
 	}
 	/* set up CPU affinity masks */
@@ -297,7 +303,7 @@ int main(int argc, char **argv)
 		return FAILURE;
 
 	/* create the groups */
-	info("Creating %d test groups\n", ngroups);
+	pi_info("Creating %d test groups\n", ngroups);
 	for (core = 0; core < num_processors; core++)
 		if (CPU_ISSET(core, &test_cpu_mask))
 			break;
@@ -320,10 +326,10 @@ int main(int argc, char **argv)
 	start = time(NULL);
 
 	/* turn loose the threads */
-	info("Releasing all threads\n");
+	pi_info("Releasing all threads\n");
 	status = pthread_barrier_wait(&all_threads_ready);
 	if (status && status != PTHREAD_BARRIER_SERIAL_THREAD) {
-		error("main: pthread_barrier_wait(all_threads_ready): 0x%x\n",
+		pi_error("main: pthread_barrier_wait(all_threads_ready): 0x%x\n",
 		      status);
 		set_shutdown_flag();
 		return FAILURE;
@@ -339,15 +345,15 @@ int main(int argc, char **argv)
 
 	/* wait for all threads to notice the shutdown flag */
 	if (have_errors == 0 && interrupted == 0) {
-		info("waiting for all threads to complete\n");
+		pi_info("waiting for all threads to complete\n");
 		status = pthread_barrier_wait(&all_threads_done);
 		if (status && status != PTHREAD_BARRIER_SERIAL_THREAD) {
-			error
+			pi_error
 			    ("main: pthread_barrier_wait(all_threads_ready): 0x%x\n",
 			     status);
 			return FAILURE;
 		}
-		info("All threads terminated!\n");
+		pi_info("All threads terminated!\n");
 		retval = SUCCESS;
 	} else
 		kill(0, SIGTERM);
@@ -367,14 +373,14 @@ setup_thread_attr(pthread_attr_t * attr, int prio, cpu_set_t * mask,
 
 	status = pthread_attr_init(attr);
 	if (status) {
-		error
+		pi_error
 		    ("setup_thread_attr: initializing thread attribute: 0x%x\n",
 		     status);
 		return FAILURE;
 	}
 	status = pthread_attr_setschedpolicy(attr, schedpolicy);
 	if (status) {
-		error
+		pi_error
 		    ("setup_thread_attr: setting attribute policy to %s: 0x%x\n",
 		     schedpolicy == SCHED_FIFO ? "SCHED_FIFO" : "SCHED_RR",
 		     status);
@@ -382,7 +388,7 @@ setup_thread_attr(pthread_attr_t * attr, int prio, cpu_set_t * mask,
 	}
 	status = pthread_attr_setinheritsched(attr, PTHREAD_EXPLICIT_SCHED);
 	if (status) {
-		error
+		pi_error
 		    ("setup_thread_attr: setting explicit scheduling inheritance: 0x%x\n",
 		     status);
 		return FAILURE;
@@ -390,13 +396,13 @@ setup_thread_attr(pthread_attr_t * attr, int prio, cpu_set_t * mask,
 	thread_param.sched_priority = prio;
 	status = pthread_attr_setschedparam(attr, &thread_param);
 	if (status) {
-		error("setup_thread_attr: setting scheduler param: 0x%x\n",
+		pi_error("setup_thread_attr: setting scheduler param: 0x%x\n",
 		      status);
 		return FAILURE;
 	}
 	status = pthread_attr_setaffinity_np(attr, sizeof(cpu_set_t), mask);
 	if (status) {
-		error("setup_thread_attr: setting affinity attribute: 0x%x\n",
+		pi_error("setup_thread_attr: setting affinity attribute: 0x%x\n",
 		      status);
 		return FAILURE;
 	}
@@ -414,14 +420,14 @@ int set_cpu_affinity(cpu_set_t * test_mask, cpu_set_t * admin_mask)
 		CPU_ZERO(test_mask);
 		CPU_SET(0, admin_mask);
 		CPU_SET(0, test_mask);
-		info("admin and test threads running on one processor\n");
+		pi_info("admin and test threads running on one processor\n");
 		return SUCCESS;
 	}
 	/* first set our main thread to run on the first
 	   scheduleable processor we can find */
 	status = sched_getaffinity(0, sizeof(cpu_set_t), &current_mask);
 	if (status) {
-		error("failed getting CPU affinity mask: 0x%x\n", status);
+		pi_error("failed getting CPU affinity mask: 0x%x\n", status);
 		return FAILURE;
 	}
 	for (i = 0; i < num_processors; i++) {
@@ -429,7 +435,7 @@ int set_cpu_affinity(cpu_set_t * test_mask, cpu_set_t * admin_mask)
 			break;
 	}
 	if (i >= num_processors) {
-		error("No schedulable CPU found for main!\n");
+		pi_error("No schedulable CPU found for main!\n");
 		return FAILURE;
 	}
 	admin_proc = i;
@@ -437,11 +443,11 @@ int set_cpu_affinity(cpu_set_t * test_mask, cpu_set_t * admin_mask)
 	CPU_SET(admin_proc, admin_mask);
 	status = sched_setaffinity(0, sizeof(cpu_set_t), admin_mask);
 	if (status) {
-		error("set_cpu_affinity: setting CPU affinity mask: 0x%x\n",
+		pi_error("set_cpu_affinity: setting CPU affinity mask: 0x%x\n",
 		      status);
 		return FAILURE;
 	}
-	info("Admin thread running on processor: %d\n", i);
+	pi_info("Admin thread running on processor: %d\n", i);
 
 	/* Set test affinity so that tests run on the non-admin processors */
 	CPU_ZERO(test_mask);
@@ -449,10 +455,10 @@ int set_cpu_affinity(cpu_set_t * test_mask, cpu_set_t * admin_mask)
 		CPU_SET(i, test_mask);
 
 	if (admin_proc + 1 == num_processors - 1)
-		info("Test threads running on processor: %ld\n",
+		pi_info("Test threads running on processor: %ld\n",
 		     num_processors - 1);
 	else
-		info("Test threads running on processors:  %d-%d\n",
+		pi_info("Test threads running on processors:  %d-%d\n",
 		     admin_proc + 1, (int)num_processors - 1);
 
 	return SUCCESS;
@@ -480,7 +486,7 @@ int watchdog_check(void)
 			if (g->inversions == g->total)
 				continue;
 			if (++g->watchdog_hits >= WATCHDOG_LIMIT) {
-				error
+				pi_error
 				    ("WATCHDOG triggered: group %d is deadlocked!\n",
 				     i);
 				failures++;
@@ -496,7 +502,7 @@ int pending_interrupt(void)
 	sigset_t pending;
 
 	if (sigpending(&pending) < 0) {
-		error("from sigpending: %s\n", strerror(errno));
+		pi_error("from sigpending: %s\n", strerror(errno));
 		return 0;
 	}
 
@@ -533,12 +539,12 @@ void *reporter(void *arg)
 	/* sleep initially to let everything get up and running */
 	status = clock_nanosleep(CLOCK_MONOTONIC, 0, &ts, NULL);
 	if (status) {
-		error("from clock_nanosleep: %s\n", strerror(status));
+		pi_error("from clock_nanosleep: %s\n", strerror(status));
 		return NULL;
 	}
 
-	debug("reporter: starting report loop\n");
-	info("Press Control-C to stop test\nCurrent Inversions: \n");
+	pi_debug("reporter: starting report loop\n");
+	pi_info("Press Control-C to stop test\nCurrent Inversions: \n");
 
 	for (;;) {
 		pthread_mutex_lock(&shutdown_mtx);
@@ -551,7 +557,7 @@ void *reporter(void *arg)
 		/* wait for our reporting interval */
 		status = clock_nanosleep(CLOCK_MONOTONIC, 0, &ts, NULL);
 		if (status) {
-			error("from clock_nanosleep: %s\n", strerror(status));
+			pi_error("from clock_nanosleep: %s\n", strerror(status));
 			break;
 		}
 
@@ -568,18 +574,18 @@ void *reporter(void *arg)
 
 		/* if we specified a duration, see if it has expired */
 		if (end && time(NULL) > end) {
-			info("duration reached (%d seconds)\n", duration);
+			pi_info("duration reached (%d seconds)\n", duration);
 			set_shutdown_flag();
 			continue;
 		}
 		/* check for a pending SIGINT */
 		if (pending_interrupt()) {
-			info("Keyboard Interrupt!\n");
+			pi_info("Keyboard Interrupt!\n");
 			break;
 		}
 		/* check watchdog stuff */
 		if ((watchdog_check())) {
-			error("reporter stopping due to watchdog event\n");
+			pi_error("reporter stopping due to watchdog event\n");
 			set_shutdown_flag();
 			break;
 		}
@@ -587,7 +593,7 @@ void *reporter(void *arg)
 		watchdog_clear();
 
 	}
-	debug("reporter: finished\n");
+	pi_debug("reporter: finished\n");
 	set_shutdown_flag();
 	return NULL;
 }
@@ -625,15 +631,15 @@ void *low_priority(void *arg)
 	allow_sigterm();
 
 	if (verify_cpu(p->cpu) != SUCCESS) {
-		error("low_priority[%d]: not bound to %ld\n", p->id, p->cpu);
+		pi_error("low_priority[%d]: not bound to %ld\n", p->id, p->cpu);
 		return NULL;
 	}
 
-	debug("low_priority[%d]: entering ready state\n", p->id);
+	pi_debug("low_priority[%d]: entering ready state\n", p->id);
 	/* wait for all threads to be ready */
 	status = pthread_barrier_wait(&all_threads_ready);
 	if (status && status != PTHREAD_BARRIER_SERIAL_THREAD) {
-		error
+		pi_error
 		    ("low_priority[%d]: pthread_barrier_wait(all_threads_ready): %x",
 		     p->id, status);
 		return NULL;
@@ -641,7 +647,7 @@ void *low_priority(void *arg)
 
 	unbounded = (p->inversions < 0);
 
-	debug("low_priority[%d]: starting inversion loop\n", p->id);
+	pi_debug("low_priority[%d]: starting inversion loop\n", p->id);
 
 	for (;;) {
 		/*
@@ -662,7 +668,7 @@ void *low_priority(void *arg)
 
 		status = pthread_barrier_wait(loop_barr);
 		if (status && status != PTHREAD_BARRIER_SERIAL_THREAD) {
-			error("%s[%d]: pthread_barrier_wait(loop): %x\n",
+			pi_error("%s[%d]: pthread_barrier_wait(loop): %x\n",
 			      __func__, p->id, status);
 			return NULL;
 		}
@@ -679,63 +685,63 @@ void *low_priority(void *arg)
 		}
 
 		/* initial state */
-		debug("low_priority[%d]: entering start wait (%d)\n", p->id,
+		pi_debug("low_priority[%d]: entering start wait (%d)\n", p->id,
 		      count++);
 		status = pthread_barrier_wait(&p->start_barrier);
 		if (status && status != PTHREAD_BARRIER_SERIAL_THREAD) {
-			error
+			pi_error
 			    ("low_priority[%d]: pthread_barrier_wait(start): %x\n",
 			     p->id, status);
 			return NULL;
 		}
 
-		debug("low_priority[%d]: claiming mutex\n", p->id);
+		pi_debug("low_priority[%d]: claiming mutex\n", p->id);
 		pthread_mutex_lock(&p->mutex);
-		debug("low_priority[%d]: mutex locked\n", p->id);
+		pi_debug("low_priority[%d]: mutex locked\n", p->id);
 
-		debug("low_priority[%d]: entering locked wait\n", p->id);
+		pi_debug("low_priority[%d]: entering locked wait\n", p->id);
 		status = pthread_barrier_wait(&p->locked_barrier);
 		if (status && status != PTHREAD_BARRIER_SERIAL_THREAD) {
-			error
+			pi_error
 			    ("low_priority[%d]: pthread_barrier_wait(locked): %x\n",
 			     p->id, status);
 			return NULL;
 		}
 		/* wait for priority boost */
-		debug("low_priority[%d]: entering elevated wait\n", p->id);
+		pi_debug("low_priority[%d]: entering elevated wait\n", p->id);
 		status = pthread_barrier_wait(&p->elevate_barrier);
 		if (status && status != PTHREAD_BARRIER_SERIAL_THREAD) {
-			error
+			pi_error
 			    ("low_priority[%d]: pthread_barrier_wait(elevate): %x\n",
 			     p->id, status);
 			return NULL;
 		}
 
 		/* release the mutex */
-		debug("low_priority[%d]: unlocking mutex\n", p->id);
+		pi_debug("low_priority[%d]: unlocking mutex\n", p->id);
 		pthread_mutex_unlock(&p->mutex);
 
 		/* finish state */
-		debug("low_priority[%d]: entering finish wait\n", p->id);
+		pi_debug("low_priority[%d]: entering finish wait\n", p->id);
 		status = pthread_barrier_wait(&p->finish_barrier);
 		if (status && status != PTHREAD_BARRIER_SERIAL_THREAD) {
-			error
+			pi_error
 			    ("low_priority[%d]: pthread_barrier_wait(elevate): %x\n",
 			     p->id, status);
 			return NULL;
 		}
 	}
 	set_shutdown_flag();
-	debug("low_priority[%d]: entering done barrier\n", p->id);
+	pi_debug("low_priority[%d]: entering done barrier\n", p->id);
 	/* wait for all threads to finish */
 	status = pthread_barrier_wait(&all_threads_done);
 	if (status && status != PTHREAD_BARRIER_SERIAL_THREAD) {
-		error
+		pi_error
 		    ("low_priority[%d]: pthread_barrier_wait(all_threads_done): %x",
 		     p->id, status);
 		return NULL;
 	}
-	debug("low_priority[%d]: exiting\n", p->id);
+	pi_debug("low_priority[%d]: exiting\n", p->id);
 	return NULL;
 }
 
@@ -752,15 +758,15 @@ void *med_priority(void *arg)
 	allow_sigterm();
 
 	if (verify_cpu(p->cpu) != SUCCESS) {
-		error("med_priority[%d]: not bound to %ld\n", p->id, p->cpu);
+		pi_error("med_priority[%d]: not bound to %ld\n", p->id, p->cpu);
 		return NULL;
 	}
 
-	debug("med_priority[%d]: entering ready state\n", p->id);
+	pi_debug("med_priority[%d]: entering ready state\n", p->id);
 	/* wait for all threads to be ready */
 	status = pthread_barrier_wait(&all_threads_ready);
 	if (status && status != PTHREAD_BARRIER_SERIAL_THREAD) {
-		error
+		pi_error
 		    ("med_priority[%d]: pthread_barrier_wait(all_threads_ready): %x",
 		     p->id, status);
 		return NULL;
@@ -768,7 +774,7 @@ void *med_priority(void *arg)
 
 	unbounded = (p->inversions < 0);
 
-	debug("med_priority[%d]: starting inversion loop\n", p->id);
+	pi_debug("med_priority[%d]: starting inversion loop\n", p->id);
 	for (;;) {
 		if (!unbounded && (p->total >= p->inversions)) {
 			set_shutdown_flag();
@@ -783,7 +789,7 @@ void *med_priority(void *arg)
 
 		status = pthread_barrier_wait(loop_barr);
 		if (status && status != PTHREAD_BARRIER_SERIAL_THREAD) {
-			error("%s[%d]: pthread_barrier_wait(loop): %x\n",
+			pi_error("%s[%d]: pthread_barrier_wait(loop): %x\n",
 			      __func__, p->id, status);
 			return NULL;
 		}
@@ -800,26 +806,26 @@ void *med_priority(void *arg)
 		}
 
 		/* start state */
-		debug("med_priority[%d]: entering start state (%d)\n", p->id,
+		pi_debug("med_priority[%d]: entering start state (%d)\n", p->id,
 		      count++);
 		status = pthread_barrier_wait(&p->start_barrier);
 		if (status && status != PTHREAD_BARRIER_SERIAL_THREAD) {
-			error
+			pi_error
 			    ("med_priority[%d]: pthread_barrier_wait(start): %x",
 			     p->id, status);
 			return NULL;
 		}
-		debug("med_priority[%d]: entering elevate state\n", p->id);
+		pi_debug("med_priority[%d]: entering elevate state\n", p->id);
 		status = pthread_barrier_wait(&p->elevate_barrier);
 		if (status && status != PTHREAD_BARRIER_SERIAL_THREAD) {
-			error ("med_priority[%d]: pthread_barrier_wait(elevate): %x", p->id, status);
+			pi_error ("med_priority[%d]: pthread_barrier_wait(elevate): %x", p->id, status);
 			return NULL;
 		}
 
-		debug("med_priority[%d]: entering finish state\n", p->id);
+		pi_debug("med_priority[%d]: entering finish state\n", p->id);
 		status = pthread_barrier_wait(&p->finish_barrier);
 		if (status && status != PTHREAD_BARRIER_SERIAL_THREAD) {
-			error
+			pi_error
 			    ("med_priority[%d]: pthread_barrier_wait(finished): %x",
 			     p->id, status);
 			return NULL;
@@ -827,19 +833,19 @@ void *med_priority(void *arg)
 	}
 	set_shutdown_flag();
 
-	debug("med_priority[%d]: entering done barrier\n", p->id);
+	pi_debug("med_priority[%d]: entering done barrier\n", p->id);
 	/* wait for all threads to finish */
 	if (have_errors == 0) {
 		status = pthread_barrier_wait(&all_threads_done);
 		if (status && status != PTHREAD_BARRIER_SERIAL_THREAD) {
-			error
+			pi_error
 			    ("med_priority[%d]: pthread_barrier_wait(all_threads_done): %x",
 			     p->id, status);
 			return NULL;
 		}
 	}
 	/* exit */
-	debug("med_priority[%d]: exiting\n", p->id);
+	pi_debug("med_priority[%d]: exiting\n", p->id);
 	return NULL;
 }
 
@@ -855,22 +861,22 @@ void *high_priority(void *arg)
 
 	allow_sigterm();
 	if (verify_cpu(p->cpu) != SUCCESS) {
-		error("high_priority[%d]: not bound to %ld\n", p->id, p->cpu);
+		pi_error("high_priority[%d]: not bound to %ld\n", p->id, p->cpu);
 		return NULL;
 	}
 
-	debug("high_priority[%d]: entering ready state\n", p->id);
+	pi_debug("high_priority[%d]: entering ready state\n", p->id);
 
 	/* wait for all threads to be ready */
 	status = pthread_barrier_wait(&all_threads_ready);
 	if (status && status != PTHREAD_BARRIER_SERIAL_THREAD) {
-		error
+		pi_error
 		    ("high_priority[%d]: pthread_barrier_wait(all_threads_ready): %x",
 		     p->id, status);
 		return NULL;
 	}
 	unbounded = (p->inversions < 0);
-	debug("high_priority[%d]: starting inversion loop\n", p->id);
+	pi_debug("high_priority[%d]: starting inversion loop\n", p->id);
 	for (;;) {
 		if (!unbounded && (p->total >= p->inversions)) {
 			set_shutdown_flag();
@@ -886,7 +892,7 @@ void *high_priority(void *arg)
 
 		status = pthread_barrier_wait(loop_barr);
 		if (status && status != PTHREAD_BARRIER_SERIAL_THREAD) {
-			error("%s[%d]: pthread_barrier_wait(loop): %x\n",
+			pi_error("%s[%d]: pthread_barrier_wait(loop): %x\n",
 			      __func__, p->id, status);
 			return NULL;
 		}
@@ -901,35 +907,35 @@ void *high_priority(void *arg)
 			}
 			pthread_mutex_unlock(&shutdown_mtx);
 		}
-		debug("high_priority[%d]: entering start state (%d)\n", p->id,
+		pi_debug("high_priority[%d]: entering start state (%d)\n", p->id,
 		      count++);
 		status = pthread_barrier_wait(&p->start_barrier);
 		if (status && status != PTHREAD_BARRIER_SERIAL_THREAD) {
-			error
+			pi_error
 			    ("high_priority[%d]: pthread_barrier_wait(start): %x",
 			     p->id, status);
 			return NULL;
 		}
 
-		debug("high_priority[%d]: entering running state\n", p->id);
+		pi_debug("high_priority[%d]: entering running state\n", p->id);
 		status = pthread_barrier_wait(&p->locked_barrier);
 		if (status && status != PTHREAD_BARRIER_SERIAL_THREAD) {
-			error
+			pi_error
 			    ("high_priority[%d]: pthread_barrier_wait(running): %x",
 			     p->id, status);
 			return NULL;
 		}
-		debug("high_priority[%d]: locking mutex\n", p->id);
+		pi_debug("high_priority[%d]: locking mutex\n", p->id);
 		pthread_mutex_lock(&p->mutex);
-		debug("high_priority[%d]: got mutex\n", p->id);
+		pi_debug("high_priority[%d]: got mutex\n", p->id);
 
-		debug("high_priority[%d]: unlocking mutex\n", p->id);
+		pi_debug("high_priority[%d]: unlocking mutex\n", p->id);
 		pthread_mutex_unlock(&p->mutex);
-		debug("high_priority[%d]: entering finish state\n", p->id);
+		pi_debug("high_priority[%d]: entering finish state\n", p->id);
 
 		status = pthread_barrier_wait(&p->finish_barrier);
 		if (status && status != PTHREAD_BARRIER_SERIAL_THREAD) {
-			error
+			pi_error
 			    ("high_priority[%d]: pthread_barrier_wait(finish): %x",
 			     status);
 			return NULL;
@@ -943,54 +949,23 @@ void *high_priority(void *arg)
 	}
 	set_shutdown_flag();
 
-	debug("high_priority[%d]: entering done barrier\n", p->id);
+	pi_debug("high_priority[%d]: entering done barrier\n", p->id);
 
 	if (have_errors == 0) {
 		/* wait for all threads to finish */
 		status = pthread_barrier_wait(&all_threads_done);
 		if (status && status != PTHREAD_BARRIER_SERIAL_THREAD) {
-			error
+			pi_error
 			    ("high_priority[%d]: pthread_barrier_wait(all_threads_done): %x",
 			     p->id, status);
 			return NULL;
 		}
 	}
 	/* exit */
-	debug("high_priority[%d]: exiting\n", p->id);
+	pi_debug("high_priority[%d]: exiting\n", p->id);
 	return NULL;
 }
 
-void error(char *fmt, ...)
-{
-	va_list ap;
-	fputs("ERROR: ", stderr);
-	va_start(ap, fmt);
-	vfprintf(stderr, fmt, ap);
-	va_end(ap);
-	have_errors = 1;
-}
-
-void info(char *fmt, ...)
-{
-	if (verbose) {
-		va_list ap;
-		va_start(ap, fmt);
-		vprintf(fmt, ap);
-		va_end(ap);
-	}
-}
-
-void debug(char *fmt, ...)
-{
-	if (debugging) {
-		va_list ap;
-		fputs("DEBUG: ", stderr);
-		va_start(ap, fmt);
-		vfprintf(stderr, fmt, ap);
-		va_end(ap);
-	}
-}
-
 void usage(void)
 {
 	printf("usage: pi_stress <options>\n");
@@ -1023,12 +998,12 @@ int block_signals(void)
 	/* mask off all signals */
 	status = sigfillset(&sigset);
 	if (status) {
-		error("setting up full signal set %s\n", strerror(status));
+		pi_error("setting up full signal set %s\n", strerror(status));
 		return FAILURE;
 	}
 	status = pthread_sigmask(SIG_BLOCK, &sigset, NULL);
 	if (status) {
-		error("setting signal mask: %s\n", strerror(status));
+		pi_error("setting signal mask: %s\n", strerror(status));
 		return FAILURE;
 	}
 	return SUCCESS;
@@ -1042,17 +1017,17 @@ int allow_sigterm(void)
 
 	status = sigemptyset(&sigset);
 	if (status) {
-		error("creating empty signal set: %s\n", strerror(status));
+		pi_error("creating empty signal set: %s\n", strerror(status));
 		return FAILURE;
 	}
 	status = sigaddset(&sigset, SIGTERM);
 	if (status) {
-		error("adding SIGTERM to signal set: %s\n", strerror(status));
+		pi_error("adding SIGTERM to signal set: %s\n", strerror(status));
 		return FAILURE;
 	}
 	status = pthread_sigmask(SIG_UNBLOCK, &sigset, NULL);
 	if (status) {
-		error("unblocking SIGTERM: %s\n", strerror(status));
+		pi_error("unblocking SIGTERM: %s\n", strerror(status));
 		return FAILURE;
 	}
 	return SUCCESS;
@@ -1064,7 +1039,7 @@ void set_shutdown_flag(void)
 	pthread_mutex_lock(&shutdown_mtx);
 	if (shutdown == 0) {
 		/* tell anyone that's looking that we're done */
-		info("setting shutdown flag\n");
+		pi_info("setting shutdown flag\n");
 		shutdown = 1;
 	}
 	pthread_mutex_unlock(&shutdown_mtx);
@@ -1082,7 +1057,7 @@ int initialize_group(struct group_parameters *group)
 	/* (make it a PI mutex) */
 	status = pthread_mutexattr_init(&mutex_attr);
 	if (status) {
-		error("initializing mutex attribute: %s\n", strerror(status));
+		pi_error("initializing mutex attribute: %s\n", strerror(status));
 		return FAILURE;
 	}
 
@@ -1090,13 +1065,13 @@ int initialize_group(struct group_parameters *group)
 	status = pthread_mutexattr_setprotocol(&mutex_attr,
 					       PTHREAD_PRIO_INHERIT);
 	if (status) {
-		error("setting mutex attribute policy: %s\n", strerror(status));
+		pi_error("setting mutex attribute policy: %s\n", strerror(status));
 		return FAILURE;
 	}
 	/* initialize the group mutex */
 	status = pthread_mutex_init(&group->mutex, &mutex_attr);
 	if (status) {
-		error("initializing mutex: %s\n", strerror(status));
+		pi_error("initializing mutex: %s\n", strerror(status));
 		return FAILURE;
 	}
 
@@ -1120,19 +1095,19 @@ int initialize_group(struct group_parameters *group)
 		return FAILURE;
 
 	if ((status = pthread_mutex_init(&group->loop_mtx, NULL)) != 0) {
-		error("pthread_mutex_init, status = %d\n", status);
+		pi_error("pthread_mutex_init, status = %d\n", status);
 		return FAILURE;
 	}
 
 	if ((status = pthread_mutex_lock(&group->loop_mtx)) != 0) {
-		error("pthread_mutex_lock, status = %d\n", status);
+		pi_error("pthread_mutex_lock, status = %d\n", status);
 		return FAILURE;
 	}
 
 	group->loop = 1;
 
 	if ((status = pthread_mutex_unlock(&group->loop_mtx)) != 0) {
-		error("pthread_mutex_unlock, status = %d\n", status);
+		pi_error("pthread_mutex_unlock, status = %d\n", status);
 		return FAILURE;
 	}
 
@@ -1149,45 +1124,45 @@ int create_group(struct group_parameters *group)
 	/* initialize group structure */
 	status = initialize_group(group);
 	if (status) {
-		error("initializing group %d\n", group->id);
+		pi_error("initializing group %d\n", group->id);
 		return FAILURE;
 	}
 
 	CPU_ZERO(&mask);
 	CPU_SET(group->cpu, &mask);
 
-	debug("group %d bound to cpu %ld\n", group->id, group->cpu);
+	pi_debug("group %d bound to cpu %ld\n", group->id, group->cpu);
 
 	/* start the low priority thread */
-	debug("creating low priority thread\n");
+	pi_debug("creating low priority thread\n");
 	if (setup_thread_attr(&thread_attr, LOW_PRIO(), &mask, policy))
 		return FAILURE;
 	status = pthread_create(&group->low_tid,
 				&thread_attr, low_priority, group);
 	if (status != 0) {
-		error("creating low_priority thread: %s\n", strerror(status));
+		pi_error("creating low_priority thread: %s\n", strerror(status));
 		return FAILURE;
 	}
 
 	/* create the medium priority thread */
-	debug("creating medium priority thread\n");
+	pi_debug("creating medium priority thread\n");
 	if (setup_thread_attr(&thread_attr, MED_PRIO(), &mask, policy))
 		return FAILURE;
 	status = pthread_create(&group->med_tid,
 				&thread_attr, med_priority, group);
 	if (status != 0) {
-		error("creating med_priority thread: %s\n", strerror(status));
+		pi_error("creating med_priority thread: %s\n", strerror(status));
 		return FAILURE;
 	}
 
 	/* create the high priority thread */
-	debug("creating high priority thread\n");
+	pi_debug("creating high priority thread\n");
 	if (setup_thread_attr(&thread_attr, HIGH_PRIO(), &mask, policy))
 		return FAILURE;
 	status = pthread_create(&group->high_tid,
 				&thread_attr, high_priority, group);
 	if (status != 0) {
-		error("creating high_priority thread: %s\n", strerror(status));
+		pi_error("creating high_priority thread: %s\n", strerror(status));
 		set_shutdown_flag();
 		return FAILURE;
 	}
@@ -1216,11 +1191,11 @@ void process_command_line(int argc, char **argv)
 			break;
 		case 'i':
 			inversions = strtol(optarg, NULL, 10);
-			info("doing %d inversion per group\n", inversions);
+			pi_info("doing %d inversion per group\n", inversions);
 			break;
 		case 'g':
 			ngroups = strtol(optarg, NULL, 10);
-			info("number of groups set to %d\n", ngroups);
+			pi_info("number of groups set to %d\n", ngroups);
 			break;
 		case 'r':
 			policy = SCHED_RR;
@@ -1297,8 +1272,8 @@ barrier_init(pthread_barrier_t * b, const pthread_barrierattr_t * attr,
 	int status;
 
 	if ((status = pthread_barrier_init(b, attr, count)) != 0) {
-		error("barrier_init: failed to initialize: %s\n", name);
-		error("status = %d\n", status);
+		pi_error("barrier_init: failed to initialize: %s\n", name);
+		pi_error("status = %d\n", status);
 		return FAILURE;
 	}
 
-- 
1.9.3

[RFC v0 6/9] rt-sched: Add sched_setattr/sched_getattr API

[Daniel Wagner]

Until we have a proper libc implementation we maintain a simple
version of it. We this new API we are able to use SCHED_DEADLINE.

This is shamelessly stolen from Dario Faggioli's libdl.

Signed-off-by: Daniel Wagner <daniel.wagner@bmw-carit.de>
Cc: Dario Faggioli <raistlin@linux.it>
---
 Makefile               |  4 +--
 src/include/rt-sched.h | 76 ++++++++++++++++++++++++++++++++++++++++++++++++++
 src/lib/rt-sched.c     | 43 ++++++++++++++++++++++++++++
 3 files changed, 121 insertions(+), 2 deletions(-)
 create mode 100644 src/include/rt-sched.h
 create mode 100644 src/lib/rt-sched.c

diff --git a/Makefile b/Makefile
index 318a5c6..87e2326 100644
--- a/Makefile
+++ b/Makefile
@@ -99,8 +99,8 @@ pip_stress: pip_stress.o librttest.a
 hackbench: hackbench.o
 	$(CC) $(CFLAGS) $(LDFLAGS) -o $@ $^ $(LIBS)
 
-librttest.a: rt-utils.o error.o rt-get_cpu.o
-	$(AR) rcs librttest.a rt-utils.o error.o rt-get_cpu.o
+librttest.a: rt-utils.o error.o rt-get_cpu.o rt-sched.o
+	$(AR) rcs librttest.a $^
 
 CLEANUP  = $(TARGETS) *.o .depend *.*~ *.orig *.rej rt-tests.spec *.d *.a
 CLEANUP += $(if $(wildcard .git), ChangeLog)
diff --git a/src/include/rt-sched.h b/src/include/rt-sched.h
new file mode 100644
index 0000000..6c45f7a
--- /dev/null
+++ b/src/include/rt-sched.h
@@ -0,0 +1,76 @@
+/*
+   rt-sched.h - sched_setattr() and sched_getattr() API
+
+   (C) Dario Faggioli <raistlin@linux.it>, 2009, 2010
+   Copyright (C) 2014 BMW Car IT GmbH, Daniel Wagner <daniel.wagner@bmw-carit.de
+
+   This program is free software; you can redistribute it and/or modify
+   it under the terms of the GNU General Public License as published by
+   the Free Software Foundation; either version 2 of the License, or
+   (at your option) any later version.
+
+   This program is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+   GNU General Public License for more details.
+
+   You should have received a copy of the GNU General Public License
+   along with this program; if not, write to the Free Software
+   Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301
+   USA */
+
+/* This file is based on Dario Faggioli's libdl. Eventually it will be
+   replaced by a proper implemenation of this API. */
+
+#ifndef __RT_SCHED_H__
+#define __RT_SCHED_H__
+
+#include <stdint.h>
+#include <sys/types.h>
+
+#ifndef SCHED_DEADLINE
+#define SCHED_DEADLINE 6
+#endif
+
+#ifdef __x86_64__
+#define __NR_sched_setattr		314
+#define __NR_sched_getattr		315
+#endif
+
+#ifdef __i386__
+#define __NR_sched_setattr		351
+#define __NR_sched_getattr		352
+#endif
+
+#ifdef __arm__
+#define __NR_sched_setattr		380
+#define __NR_sched_getattr		381
+#endif
+
+struct sched_attr {
+	uint32_t size;
+	uint32_t sched_policy;
+	uint64_t sched_flags;
+
+	/* SCHED_NORMAL, SCHED_BATCH */
+	int32_t sched_nice;
+
+	/* SCHED_FIFO, SCHED_RR */
+	uint32_t sched_priority;
+
+	/* SCHED_DEADLINE */
+	uint64_t sched_runtime;
+	uint64_t sched_deadline;
+	uint64_t sched_period;
+};
+
+int sched_setattr(pid_t pid,
+		  const struct sched_attr *attr,
+		  unsigned int flags);
+
+int sched_getattr(pid_t pid,
+		  struct sched_attr *attr,
+		  unsigned int size,
+		  unsigned int flags);
+
+#endif /* __RT_SCHED_H__ */
diff --git a/src/lib/rt-sched.c b/src/lib/rt-sched.c
new file mode 100644
index 0000000..4a8e3c4
--- /dev/null
+++ b/src/lib/rt-sched.c
@@ -0,0 +1,43 @@
+/*
+   rt-sched.h - sched_setattr() and sched_getattr() API
+
+   (C) Dario Faggioli <raistlin@linux.it>, 2009, 2010
+   Copyright (C) 2014 BMW Car IT GmbH, Daniel Wagner <daniel.wagner@bmw-carit.de
+
+   This program is free software; you can redistribute it and/or modify
+   it under the terms of the GNU General Public License as published by
+   the Free Software Foundation; either version 2 of the License, or
+   (at your option) any later version.
+
+   This program is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+   GNU General Public License for more details.
+
+   You should have received a copy of the GNU General Public License
+   along with this program; if not, write to the Free Software
+   Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301
+   USA */
+
+/* This file is based on Dario Faggioli's libdl. Eventually it will be
+   replaced by a proper implemenation of this API. */
+
+#include <unistd.h>
+#include <sys/syscall.h>
+
+#include "rt-sched.h"
+
+int sched_setattr(pid_t pid,
+		  const struct sched_attr *attr,
+		  unsigned int flags)
+{
+	return syscall(__NR_sched_setattr, pid, attr, flags);
+}
+
+int sched_getattr(pid_t pid,
+		  struct sched_attr *attr,
+		  unsigned int size,
+		  unsigned int flags)
+{
+        return syscall(__NR_sched_getattr, pid, attr, size, flags);
+}
-- 
1.9.3

[RFC v0 7/9] rt-utils: Add helper to parse/print scheduling policies

[Daniel Wagner]

Signed-off-by: Daniel Wagner <daniel.wagner@bmw-carit.de>
---
 src/include/rt-utils.h |  5 +++++
 src/lib/rt-utils.c     | 38 ++++++++++++++++++++++++++++++++++++++
 2 files changed, 43 insertions(+)

diff --git a/src/include/rt-utils.h b/src/include/rt-utils.h
index 4a7d01f..37f46c8 100644
--- a/src/include/rt-utils.h
+++ b/src/include/rt-utils.h
@@ -1,6 +1,8 @@
 #ifndef __RT_UTILS_H
 #define __RT_UTILS_H
 
+#include <stdint.h>
+
 #define _STR(x) #x
 #define STR(x) _STR(x)
 #define MAX_PATH 256
@@ -17,4 +19,7 @@ int event_disable(char *event);
 int event_enable_all(void);
 int event_disable_all(void);
 
+const char *policy_to_string(int policy);
+uint32_t string_to_policy(const char *str);
+
 #endif	/* __RT_UTILS.H */
diff --git a/src/lib/rt-utils.c b/src/lib/rt-utils.c
index f4da4b3..df522fe 100644
--- a/src/lib/rt-utils.c
+++ b/src/lib/rt-utils.c
@@ -17,6 +17,7 @@
 #include <sys/stat.h>
 #include <unistd.h>
 #include "rt-utils.h"
+#include "rt-sched.h"
 #include "error.h"
 
 static char debugfileprefix[MAX_PATH];
@@ -273,3 +274,40 @@ int check_privs(void)
 	return sched_setscheduler(0, policy, &old_param);
 }
 
+const char *policy_to_string(int policy)
+{
+	switch (policy) {
+	case SCHED_OTHER:
+		return "SCHED_OTHER";
+	case SCHED_FIFO:
+		return "SCHED_FIFO";
+	case SCHED_RR:
+		return "SCHED_RR";
+	case SCHED_BATCH:
+		return "SCHED_BATCH";
+	case SCHED_IDLE:
+		return "SCHED_IDLE";
+	case SCHED_DEADLINE:
+		return "SCHED_DEADLINE";
+	}
+
+	return "unknown";
+}
+
+uint32_t string_to_policy(const char *str)
+{
+	if (!strcmp(str, "other"))
+		return SCHED_OTHER;
+	else if (!strcmp(str, "fifo"))
+		return SCHED_FIFO;
+	else if (!strcmp(str, "rr"))
+		return SCHED_RR;
+	else if (!strcmp(str, "batch"))
+		return SCHED_BATCH;
+	else if (!strcmp(str, "idle"))
+		return SCHED_IDLE;
+	else if (!strcmp(str, "deadline"))
+		return SCHED_DEADLINE;
+
+	return 0;
+}
-- 
1.9.3

[RFC v0 8/9] rt-utils: Add gettid()

[Daniel Wagner]

We still lack a gettid implemenation from libc.

Signed-off-by: Daniel Wagner <daniel.wagner@bmw-carit.de>
---
 src/include/rt-utils.h | 2 ++
 src/lib/rt-utils.c     | 6 ++++++
 2 files changed, 8 insertions(+)

diff --git a/src/include/rt-utils.h b/src/include/rt-utils.h
index 37f46c8..ef0f6ac 100644
--- a/src/include/rt-utils.h
+++ b/src/include/rt-utils.h
@@ -22,4 +22,6 @@ int event_disable_all(void);
 const char *policy_to_string(int policy);
 uint32_t string_to_policy(const char *str);
 
+pid_t gettid(void);
+
 #endif	/* __RT_UTILS.H */
diff --git a/src/lib/rt-utils.c b/src/lib/rt-utils.c
index df522fe..3882d23 100644
--- a/src/lib/rt-utils.c
+++ b/src/lib/rt-utils.c
@@ -16,6 +16,7 @@
 #include <sys/types.h>
 #include <sys/stat.h>
 #include <unistd.h>
+#include <sys/syscall.h> /* For SYS_gettid definitions */
 #include "rt-utils.h"
 #include "rt-sched.h"
 #include "error.h"
@@ -311,3 +312,8 @@ uint32_t string_to_policy(const char *str)
 
 	return 0;
 }
+
+pid_t gettid(void)
+{
+	return syscall(SYS_gettid);
+}
-- 
1.9.3

[RFC v0 9/9] pi_stress: Store schedule attributes per thread

[Daniel Wagner]

Currently, the scheduling class is configured on a global
level. It is possible to run the test either with SCHED_FIFO
or SCHED_RR. All threads run then with the same configuration
except sched_priority is different.

By storing the scheduling attributes per thread we will be able
to use different scheduler classes at the same time. The aim is
to use SCHED_DEADLINE for the high priority thread.

First thing to get there is to introduce low_sa, med_sa, high_sa
and admin_sa. They are configured using the global policy variable
on default. Either using SCHED_FIFO or SCHED_RR. The user
can though use --sched command line options to configure each
thread seperately. E.g.

Starting PI Stress Test
Number of thread groups: 1
Duration of test run: infinite
Number of inversions per group: unlimited
     Admin thread SCHED_FIFO priority 4
1 groups of 3 threads will be created
      High thread SCHED_DEADLINE runtime 100000 deadline 200000 period 200000
       Med thread SCHED_FIFO priority 2
       Low thread SCHED_FIFO priority 1
Current Inversions: 2446249
Stopping test
Terminated

Signed-off-by: Daniel Wagner <daniel.wagner@bmw-carit.de>
---
 src/pi_tests/pi_stress.c | 245 ++++++++++++++++++++++++++++++++++++++++-------
 1 file changed, 211 insertions(+), 34 deletions(-)

diff --git a/src/pi_tests/pi_stress.c b/src/pi_tests/pi_stress.c
index d834b86..46058e2 100644
--- a/src/pi_tests/pi_stress.c
+++ b/src/pi_tests/pi_stress.c
@@ -53,7 +53,14 @@
 #include <sys/types.h>
 #include <sys/mman.h>
 #include <sys/wait.h>
+#include <sys/syscall.h>
 #include <termios.h>
+#include <stdint.h>
+#include <inttypes.h>
+#include <limits.h>
+
+#include "rt-sched.h"
+#include "rt-utils.h"
 
 #include "error.h"
 
@@ -145,6 +152,7 @@ struct option options[] = {
 	{"groups", required_argument, NULL, 'g'},
 	{"inversions", required_argument, NULL, 'i'},
 	{"rr", no_argument, NULL, 'r'},
+	{"sched", required_argument, NULL, 's'},
 	{"uniprocessor", no_argument, NULL, 'u'},
 	{"prompt", no_argument, NULL, 'p'},
 	{"debug", no_argument, NULL, 'd'},
@@ -154,15 +162,6 @@ struct option options[] = {
 	{NULL, 0, NULL, 0},
 };
 
-/* max priority for the scheduling policy */
-int prio_min;
-
-/* define priorities for the threads */
-#define MAIN_PRIO() (prio_min + 3)
-#define HIGH_PRIO() (prio_min + 2)
-#define MED_PRIO()  (prio_min + 1)
-#define LOW_PRIO()  (prio_min + 0)
-
 #define NUM_TEST_THREADS 3
 #define NUM_ADMIN_THREADS 1
 
@@ -173,6 +172,19 @@ cpu_set_t test_cpu_mask, admin_cpu_mask;
 
 int policy = SCHED_FIFO;
 
+/* scheduling attributes per thread */
+struct sched_attr low_sa;
+struct sched_attr med_sa;
+struct sched_attr high_sa;
+struct sched_attr admin_sa;
+
+#define SA_INIT_LOW	(1 << 0)
+#define SA_INIT_MED	(1 << 1)
+#define SA_INIT_HIGH	(1 << 2)
+#define SA_INIT_ADMIN	(1 << 3)
+
+unsigned int sa_initialized;
+
 struct group_parameters {
 
 	/* group id (index) */
@@ -226,8 +238,8 @@ void *med_priority(void *arg);
 void *high_priority(void *arg);
 void *reporter(void *arg);
 void *watchdog(void *arg);
-int setup_thread_attr(pthread_attr_t * attr, int prio, cpu_set_t * mask,
-		      int schedpolicy);
+int setup_thread_attr(pthread_attr_t * attr, struct sched_attr * sa,
+		      cpu_set_t * mask);
 int set_cpu_affinity(cpu_set_t * test_mask, cpu_set_t * admin_mask);
 void process_command_line(int argc, char **argv);
 void usage(void);
@@ -242,6 +254,8 @@ void summary(void);
 void wait_for_termination(void);
 int barrier_init(pthread_barrier_t * b, const pthread_barrierattr_t * attr,
 		 unsigned count, const char *name);
+void setup_sched_attr(struct sched_attr *attr, int policy, int prio);
+void setup_sched_config(int policy);
 
 int main(int argc, char **argv)
 {
@@ -261,9 +275,13 @@ int main(int argc, char **argv)
 	/* calculate the number of inversion groups to run */
 	ngroups = num_processors == 1 ? 1 : num_processors - 1;
 
+
 	/* process command line arguments */
 	process_command_line(argc, argv);
 
+	/* set default sched attributes */
+	setup_sched_config(policy);
+
 	/* lock memory */
 	if (lockall)
 		if (mlockall(MCL_CURRENT | MCL_FUTURE) == -1) {
@@ -271,9 +289,9 @@ int main(int argc, char **argv)
 			return FAILURE;
 		}
 	/* boost main's priority (so we keep running) :) */
-	prio_min = sched_get_priority_min(policy);
-	thread_param.sched_priority = MAIN_PRIO();
-	status = pthread_setschedparam(pthread_self(), policy, &thread_param);
+	thread_param.sched_priority = admin_sa.sched_priority;
+	status = pthread_setschedparam(pthread_self(), admin_sa.sched_policy,
+				       &thread_param);
 	if (status) {
 		pi_error("main: boosting to max priority: 0x%x\n", status);
 		return FAILURE;
@@ -365,8 +383,8 @@ int main(int argc, char **argv)
 }
 
 int
-setup_thread_attr(pthread_attr_t * attr, int prio, cpu_set_t * mask,
-		  int schedpolicy)
+setup_thread_attr(pthread_attr_t * attr, struct sched_attr * sa,
+		  cpu_set_t * mask)
 {
 	int status;
 	struct sched_param thread_param;
@@ -378,11 +396,23 @@ setup_thread_attr(pthread_attr_t * attr, int prio, cpu_set_t * mask,
 		     status);
 		return FAILURE;
 	}
-	status = pthread_attr_setschedpolicy(attr, schedpolicy);
+	status = pthread_attr_setaffinity_np(attr, sizeof(cpu_set_t), mask);
+	if (status) {
+		pi_error("setup_thread_attr: setting affinity attribute: 0x%x\n",
+		      status);
+		return FAILURE;
+	}
+
+	/* The pthread API does not yet support SCHED_DEADLINE, defer the
+	 * thread configuration to setup_thread() */
+	if (sa->sched_policy == SCHED_DEADLINE)
+		return SUCCESS;
+
+	status = pthread_attr_setschedpolicy(attr, sa->sched_policy);
 	if (status) {
 		pi_error
 		    ("setup_thread_attr: setting attribute policy to %s: 0x%x\n",
-		     schedpolicy == SCHED_FIFO ? "SCHED_FIFO" : "SCHED_RR",
+		     policy_to_string(sa->sched_policy),
 		     status);
 		return FAILURE;
 	}
@@ -393,19 +423,13 @@ setup_thread_attr(pthread_attr_t * attr, int prio, cpu_set_t * mask,
 		     status);
 		return FAILURE;
 	}
-	thread_param.sched_priority = prio;
+	thread_param.sched_priority = sa->sched_priority;
 	status = pthread_attr_setschedparam(attr, &thread_param);
 	if (status) {
 		pi_error("setup_thread_attr: setting scheduler param: 0x%x\n",
 		      status);
 		return FAILURE;
 	}
-	status = pthread_attr_setaffinity_np(attr, sizeof(cpu_set_t), mask);
-	if (status) {
-		pi_error("setup_thread_attr: setting affinity attribute: 0x%x\n",
-		      status);
-		return FAILURE;
-	}
 	return SUCCESS;
 }
 
@@ -859,6 +883,16 @@ void *high_priority(void *arg)
 	pthread_mutex_t *loop_mtx = &p->loop_mtx;
 	int *loop = &p->loop;
 
+	if (high_sa.sched_policy == SCHED_DEADLINE) {
+		status = sched_setattr(gettid(), &high_sa, 0);
+		if (status < 0) {
+			pi_error
+			    ("high_priority[%d]: sched_setattr(dl): %x\n",
+			    p->id, status);
+			return NULL;
+		}
+	}
+
 	allow_sigterm();
 	if (verify_cpu(p->cpu) != SUCCESS) {
 		pi_error("high_priority[%d]: not bound to %ld\n", p->id, p->cpu);
@@ -980,6 +1014,10 @@ void usage(void)
 	    ("\t--inversions=<n>- number of inversions per group [infinite]\n");
 	printf("\t--report=<path>\t- output to file [/dev/null]\n");
 	printf("\t--rr\t\t- use SCHED_RR for test threads [SCHED_FIFO]\n");
+	printf("\t--sched\t\t- scheduling options per thread type:\n");
+	printf("\t\tid=[high|med|low]\t\t\t- select thread\n");
+	printf("\t\t,policy=[fifo,rr],priority=<n>\t\t- SCHED_FIFO or SCHED_RR\n");
+	printf("\t\t,policy=deadline,runtime=<n>,deadline=<n>,period=<n>\t- SCHED_DEADLINE\n");
 	printf("\t--prompt\t- prompt before starting the test\n");
 	printf
 	    ("\t--uniprocessor\t- force all threads to run on one processor\n");
@@ -1135,7 +1173,7 @@ int create_group(struct group_parameters *group)
 
 	/* start the low priority thread */
 	pi_debug("creating low priority thread\n");
-	if (setup_thread_attr(&thread_attr, LOW_PRIO(), &mask, policy))
+	if (setup_thread_attr(&thread_attr, &low_sa, &mask))
 		return FAILURE;
 	status = pthread_create(&group->low_tid,
 				&thread_attr, low_priority, group);
@@ -1146,7 +1184,7 @@ int create_group(struct group_parameters *group)
 
 	/* create the medium priority thread */
 	pi_debug("creating medium priority thread\n");
-	if (setup_thread_attr(&thread_attr, MED_PRIO(), &mask, policy))
+	if (setup_thread_attr(&thread_attr, &med_sa, &mask))
 		return FAILURE;
 	status = pthread_create(&group->med_tid,
 				&thread_attr, med_priority, group);
@@ -1157,7 +1195,7 @@ int create_group(struct group_parameters *group)
 
 	/* create the high priority thread */
 	pi_debug("creating high priority thread\n");
-	if (setup_thread_attr(&thread_attr, HIGH_PRIO(), &mask, policy))
+	if (setup_thread_attr(&thread_attr, &high_sa, &mask))
 		return FAILURE;
 	status = pthread_create(&group->high_tid,
 				&thread_attr, high_priority, group);
@@ -1169,6 +1207,99 @@ int create_group(struct group_parameters *group)
 	return SUCCESS;
 }
 
+unsigned long parse_unsigned(const char *str)
+{
+	unsigned long n;
+	char *p;
+
+	errno = 0;
+	n = strtoul(str, &p, 10);
+
+	if ((errno == ERANGE && n == ULONG_MAX)
+			|| (errno != 0 && n == 0)) {
+		pi_error("parsing number failed: %s\n", str);
+		exit(EXIT_FAILURE);
+	}
+
+	return n;
+}
+
+long parse_signed(const char *str)
+{
+	long n;
+	char *p;
+
+	errno = 0;
+	n = strtol(str, &p, 10);
+
+	if ((errno == ERANGE && (n == LONG_MAX || n == LONG_MIN))
+			|| (errno != 0 && n == 0)) {
+		pi_error("parsing number failed: %s\n", str);
+		exit(EXIT_FAILURE);
+	}
+
+	return n;
+}
+
+int process_sched_line(const char *arg)
+{
+	char *buf, *k, *v;
+	const char del[] = ",=";
+	struct sched_attr sa = { 0, };
+	char *id = NULL;
+	int retval = SUCCESS;
+
+	buf = strdupa(arg);
+
+	k = strsep(&buf, del);
+	while (k) {
+		v = strsep(&buf, del);
+		if (!v)
+			break;
+
+		if (!strcmp(k, "id"))
+			id = v;
+		else if (!strcmp(k, "policy"))
+			sa.sched_policy = string_to_policy(v);
+		else if (!strcmp(k, "nice"))
+			sa.sched_nice = parse_signed(v);
+		else if (!strcmp(k, "priority"))
+			sa.sched_priority = parse_unsigned(v);
+		else if (!strcmp(k, "runtime"))
+			sa.sched_runtime = parse_unsigned(v);
+		else if (!strcmp(k, "deadline"))
+			sa.sched_deadline = parse_unsigned(v);
+		else if (!strcmp(k, "period"))
+			sa.sched_period = parse_unsigned(v);
+
+		k = strsep(&buf, del);
+	}
+
+	if (!id) {
+		free(buf);
+		return FAILURE;
+	}
+
+	/* We do not validate the options, instead we pass all garbage
+	 * to the kernel and see what's happening */
+
+	if (!strcmp(id, "low")) {
+		memcpy(&low_sa, &sa, sizeof(struct sched_attr));
+		sa_initialized |= SA_INIT_LOW;
+	} else if (!strcmp(id, "med")) {
+		memcpy(&med_sa, &sa, sizeof(struct sched_attr));
+		sa_initialized |= SA_INIT_MED;
+	} else if (!strcmp(id, "high")) {
+		memcpy(&high_sa, &sa, sizeof(struct sched_attr));
+		sa_initialized |= SA_INIT_HIGH;
+	} else {
+		retval = FAILURE;
+	}
+
+	free(buf);
+	return retval;
+}
+
 void process_command_line(int argc, char **argv)
 {
 	int opt;
@@ -1200,6 +1331,10 @@ void process_command_line(int argc, char **argv)
 		case 'r':
 			policy = SCHED_RR;
 			break;
+		case 's':
+			if (process_sched_line(optarg))
+				pi_error("ignoring invalid options '%s'\n", optarg);
+			break;
 		case 'p':
 			prompt = 1;
 			break;
@@ -1231,6 +1366,27 @@ unsigned long total_inversions(void)
 	return total;
 }
 
+void print_sched_attr(const char *name, struct sched_attr * sa)
+{
+	printf("    %6s thread", name);
+	printf(" %s", policy_to_string(sa->sched_policy));
+
+	switch(sa->sched_policy) {
+	case SCHED_OTHER:
+		printf(" nice %d\n", sa->sched_nice);
+		break;
+	case SCHED_FIFO:
+	case SCHED_RR:
+		printf(" priority %d\n", sa->sched_priority);
+		break;
+	case SCHED_DEADLINE:
+		printf(" runtime %" PRIu64 " deadline %" PRIu64 " period %" PRIu64 "\n",
+			sa->sched_runtime, sa->sched_deadline,
+			sa->sched_period);
+		break;
+	}
+}
+
 void banner(void)
 {
 	if (quiet)
@@ -1246,13 +1402,12 @@ void banner(void)
 		printf("Number of inversions per group: unlimited\n");
 	else
 		printf("Number of inversions per group: %d\n", inversions);
-	printf("Test threads using scheduler policy: %s\n",
-	       policy == SCHED_FIFO ? "SCHED_FIFO" : "SCHED_RR");
-	printf("    Admin thread priority:  %d\n", MAIN_PRIO());
+	print_sched_attr("Admin", &admin_sa);
 	printf("%d groups of 3 threads will be created\n", ngroups);
-	printf("    High thread priority:   %d\n", HIGH_PRIO());
-	printf("    Med  thread priority:   %d\n", MED_PRIO());
-	printf("    Low thread priority:    %d\n\n", LOW_PRIO());
+	print_sched_attr("High", &high_sa);
+	print_sched_attr("Med", &med_sa);
+	print_sched_attr("Low", &low_sa);
+	printf("\n");
 }
 
 void summary(void)
@@ -1279,3 +1434,25 @@ barrier_init(pthread_barrier_t * b, const pthread_barrierattr_t * attr,
 
 	return SUCCESS;
 }
+
+void setup_sched_attr(struct sched_attr *attr, int policy, int prio)
+{
+	attr->sched_policy = policy;
+	attr->sched_priority = prio;
+}
+
+void setup_sched_config(int policy)
+{
+	int prio_min;
+
+	prio_min = sched_get_priority_min(policy);
+
+	if (!(sa_initialized & SA_INIT_LOW))
+		setup_sched_attr(&low_sa,   policy, prio_min + 0);
+	if (!(sa_initialized & SA_INIT_MED))
+		setup_sched_attr(&med_sa,   policy, prio_min + 1);
+	if (!(sa_initialized & SA_INIT_HIGH))
+		setup_sched_attr(&high_sa,  policy, prio_min + 2);
+	if (!(sa_initialized & SA_INIT_ADMIN))
+		setup_sched_attr(&admin_sa, policy, prio_min + 3);
+}
-- 
1.9.3