[LTP] [PATCH 1/2] sched_setparam/9-1: fix hang on -rt kernel

[LTP] [PATCH 1/2] sched_setparam/9-1: fix hang on -rt kernel

[Jan Stancek]

As mentioned in [1], the testcase hangs on -rt kernel,
because alarm(2) will never fire. The testcase runs with
FIFO scheduling of higher priority than ksoftirqd.

This patch changes testcase in following way:
1. In good case, where child process preempts parent,
   child is able to detect this condition and terminate.
2. In bad case, where preemption fails, testcase doesn't
   rely on softirq (alarm), but hrtimer (time).
3. removes unnecessary sleep

[1] https://lkml.org/lkml/2006/1/10/481

Reported-by: Jiri Kastner <jkastner@redhat.com>
Signed-off-by: Jan Stancek <jstancek@redhat.com>
---
 .../conformance/interfaces/sched_setparam/9-1.c    | 48 ++++++++++++++--------
 1 file changed, 32 insertions(+), 16 deletions(-)

diff --git a/testcases/open_posix_testsuite/conformance/interfaces/sched_setparam/9-1.c b/testcases/open_posix_testsuite/conformance/interfaces/sched_setparam/9-1.c
index ed90cee..354e03e 100644
--- a/testcases/open_posix_testsuite/conformance/interfaces/sched_setparam/9-1.c
+++ b/testcases/open_posix_testsuite/conformance/interfaces/sched_setparam/9-1.c
@@ -31,14 +31,15 @@
  *      not, the test fail.
  */
 #define _GNU_SOURCE
+#include <errno.h>
 #include <sched.h>
 #include <stdio.h>
 #include <signal.h>
+#include <stdlib.h>
 #include <sys/ipc.h>
 #include <sys/shm.h>
+#include <time.h>
 #include <unistd.h>
-#include <stdlib.h>
-#include <errno.h>
 #include "posixtest.h"
 #include "affinity.h"
 
@@ -55,6 +56,7 @@
 
 static int nb_cpu;
 static int *shmptr;
+static int mean_prio;
 
 static int get_ncpu(void)
 {
@@ -85,6 +87,7 @@ static int get_ncpu(void)
 static void child_process(void)
 {
 	struct sched_param param;
+	time_t t1, t2;
 
 	param.sched_priority = sched_get_priority_max(SCHED_FIFO);
 	if (sched_setparam(getpid(), &param) != 0) {
@@ -92,20 +95,33 @@ static void child_process(void)
 		return;
 	}
 
-	/* to avoid blocking */
-	alarm(2);
-	while (1) ;
+	t1 = time(NULL);
+	do {
+		t2 = time(NULL);
+	} while (difftime(t2, t1) <= 2);
 }
 
 static void test_process(void)
 {
-	/* to avoid blocking */
-	alarm(2);
-
-	while (1) {
-		(*shmptr)++;
+	struct sched_param param;
+	time_t t1, t2;
+
+	t1 = time(NULL);
+	do {
+		sched_getparam(getpid(), &param);
+		(*shmptr) = param.sched_priority;
+		/* if we can see that our priority has changed
+		 * that means we preempted parent, so we are done */
+		if ((*shmptr) == mean_prio)
+			break;
+
+		t2 = time(NULL);
+		/* immediately after parent forks us, we has same
+		 * priority and compete with parent for same CPU,
+		 * give parent chance to run and boost our priority */
 		sched_yield();
-	}
+	} while (difftime(t2, t1) <= 2);
+	exit(0);
 }
 
 static void kill_children(int *child_pid)
@@ -133,6 +149,8 @@ int main(void)
 		nb_cpu = 1;
 	}
 
+	mean_prio = (sched_get_priority_min(SCHED_FIFO) +
+		sched_get_priority_max(SCHED_FIFO)) / 2;
 	child_pid = malloc(nb_cpu * sizeof(int));
 
 	key = ftok("conformance/interfaces/sched_setparam/9-1.c", 1234);
@@ -194,11 +212,9 @@ int main(void)
 		return PTS_UNRESOLVED;
 	}
 
-	sleep(1);
-
-	param.sched_priority = (sched_get_priority_min(SCHED_FIFO) +
-				sched_get_priority_max(SCHED_FIFO)) / 2;
-
+	/* parent runs, which means test_process() gave up cpu,
+	 * boost its priority and check it preempted parent */
+	param.sched_priority = mean_prio;
 	oldcount = *shmptr;
 	if (sched_setparam(child_pid[i], &param) != 0) {
 		perror("An error occurs when calling sched_setparam()");
-- 
1.8.3.1


------------------------------------------------------------------------------
Dive into the World of Parallel Programming The Go Parallel Website, sponsored
by Intel and developed in partnership with Slashdot Media, is your hub for all
things parallel software development, from weekly thought leadership blogs to
news, videos, case studies, tutorials and more. Take a look and join the 
conversation now. http://goparallel.sourceforge.net/
_______________________________________________
Ltp-list mailing list
Ltp-list@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/ltp-list

[LTP] [PATCH 2/2] sched_setparam/10-1: remove testcase

[Jan Stancek]

As in 9-1, this hangs on -rt kernel, because alarm(2)
never fires.

In case it runs on single cpu system, it's pretty
much same as 9-1: child process of higher priority
preempts parent.

In case it runs on 2+ CPUs, it doesn't matter that
one child is able to preempt parent, shared value
is going to be increased regardless in all other
children.

Signed-off-by: Jan Stancek <jstancek@redhat.com>
---
 .../conformance/interfaces/sched_setparam/10-1.c   | 180 ---------------------
 1 file changed, 180 deletions(-)
 delete mode 100644 testcases/open_posix_testsuite/conformance/interfaces/sched_setparam/10-1.c

diff --git a/testcases/open_posix_testsuite/conformance/interfaces/sched_setparam/10-1.c b/testcases/open_posix_testsuite/conformance/interfaces/sched_setparam/10-1.c
deleted file mode 100644
index 15dfe41..0000000
--- a/testcases/open_posix_testsuite/conformance/interfaces/sched_setparam/10-1.c
+++ /dev/null
@@ -1,180 +0,0 @@
-/*
- *  This program is free software; you can redistribute it and/or modify
- *  it under the terms of the GNU General Public License version 2.
- *
- *  This program is distributed in the hope that it will be useful,
- *  but WITHOUT ANY WARRANTY; without even the implied warranty of
- *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
- *  GNU General Public License for more details.
- *
- *
- * Test that the process that is the head of the highest priority list preempt
- * the process calling sched_setparam() when the calling process sets its own
- * priority lower than that of one or more other non-empty process lists.
- *
- * There is no portable way to get the number of CPUs but the test should work
- * for most of UNIX system (including but not limited to: Linux, Solaris, AIX,
- * HPUX, *BSD).
- * This test used shared memory.
- * Steps:
- *   1. Create a share memory segment.
- *   2. Change the policy to SCHED_FIFO.
- *   3. Create nb_cpu children processes which shall have the same priority
- *      and policy than the father (cf specs of fork).
- *   4. Call sched_setparam with a priority smaller than those of children.
- *   5. Check if the shared value has been changed by a child process. If not,
- *      the test fail.
- */
-
-#include <sched.h>
-#include <stdio.h>
-#include <signal.h>
-#include <sys/ipc.h>
-#include <sys/shm.h>
-#include <unistd.h>
-#include <stdlib.h>
-#include <errno.h>
-#include "posixtest.h"
-
-#ifdef BSD
-#include <sys/types.h>
-#include <sys/param.h>
-#include <sys/sysctl.h>
-#endif
-
-#ifdef HPUX
-#include <sys/param.h>
-#include <sys/pstat.h>
-#endif
-
-static int nb_cpu;
-static int *shmptr;
-
-static int get_ncpu(void)
-{
-	int ncpu = -1;
-
-	/* This syscall is not POSIX but it should work on many system */
-#ifdef _SC_NPROCESSORS_ONLN
-	ncpu = sysconf(_SC_NPROCESSORS_ONLN);
-#else
-#ifdef BSD
-	int mib[2];
-	size_t len = sizeof(ncpu);
-	mib[0] = CTL_HW;
-	mib[1] = HW_NCPU;
-	sysctl(mib, 2, &ncpu, &len, NULL, 0);
-#else
-#ifdef HPUX
-	struct pst_dynamic psd;
-	pstat_getdynamic(&psd, sizeof(psd), 1, 0);
-	ncpu = (int)psd.psd_proc_cnt;
-#endif /* HPUX */
-#endif /* BSD */
-#endif /* _SC_NPROCESSORS_ONLN */
-
-	return ncpu;
-}
-
-static void child_process(void)
-{
-	alarm(2);
-
-	while (1) {
-		(*shmptr)++;
-		sched_yield();
-	}
-}
-
-static void kill_children(int *child_pid)
-{
-	int i;
-
-	for (i = 0; i < nb_cpu; i++) {
-		kill(child_pid[i], SIGTERM);
-	}
-}
-
-int main(void)
-{
-	int *child_pid, oldcount, newcount, shm_id, i, j;
-	struct sched_param param;
-	key_t key;
-
-	nb_cpu = get_ncpu();
-
-	if (nb_cpu == -1) {
-		printf("Can not get the number of CPUs of your machines.\n");
-		return PTS_UNRESOLVED;
-	}
-
-	child_pid = malloc(nb_cpu * sizeof(int));
-
-	key = ftok("conformance/interfaces/sched_setparam/10-1.c", 1234);
-	shm_id = shmget(key, sizeof(int), IPC_CREAT | 0600);
-	if (shm_id < 0) {
-		perror("An error occurs when calling shmget()");
-		return PTS_UNRESOLVED;
-	}
-
-	shmptr = shmat(shm_id, 0, 0);
-	if (shmptr == (void *)-1) {
-		perror("An error occurs when calling shmat()");
-		return PTS_UNRESOLVED;
-	}
-	*shmptr = 0;
-
-	param.sched_priority = sched_get_priority_max(SCHED_FIFO);
-
-	if (sched_setscheduler(getpid(), SCHED_FIFO, &param) != 0) {
-		if (errno == EPERM) {
-			printf
-			    ("This process does not have the permission to set its own scheduling "
-			     "parameter.\nTry to launch this test as root\n");
-		} else {
-			perror
-			    ("An error occurs when calling sched_setscheduler()");
-		}
-		return PTS_UNRESOLVED;
-	}
-
-	for (i = 0; i < nb_cpu; i++) {
-		child_pid[i] = fork();
-		if (child_pid[i] == -1) {
-			perror("An error occurs when calling fork()");
-			for (j = 0; j < i; j++) {
-				kill(child_pid[j], SIGTERM);
-			}
-			return PTS_UNRESOLVED;
-		} else if (child_pid[i] == 0) {
-
-			child_process();
-
-			printf("This code should not be executed.\n");
-			return PTS_UNRESOLVED;
-		}
-	}
-
-	sleep(1);
-
-	param.sched_priority--;
-
-	oldcount = *shmptr;
-	if (sched_setparam(getpid(), &param) != 0) {
-		perror("An error occurs when calling sched_setparam()");
-		kill_children(child_pid);
-		return PTS_UNRESOLVED;
-	}
-	newcount = *shmptr;
-
-	if (newcount == oldcount) {
-		printf
-		    ("The calling process does not relinquish the processor\n");
-		kill_children(child_pid);
-		return PTS_FAIL;
-	}
-
-	printf("Test PASSED\n");
-	kill_children(child_pid);
-	return PTS_PASS;
-}
-- 
1.8.3.1


------------------------------------------------------------------------------
Dive into the World of Parallel Programming The Go Parallel Website, sponsored
by Intel and developed in partnership with Slashdot Media, is your hub for all
things parallel software development, from weekly thought leadership blogs to
news, videos, case studies, tutorials and more. Take a look and join the 
conversation now. http://goparallel.sourceforge.net/
_______________________________________________
Ltp-list mailing list
Ltp-list@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/ltp-list

Re: [LTP] [PATCH 2/2] sched_setparam/10-1: remove testcase

[Cyril Hrubis]

Hi!
> As in 9-1, this hangs on -rt kernel, because alarm(2)
> never fires.
> 
> In case it runs on single cpu system, it's pretty
> much same as 9-1: child process of higher priority
> preempts parent.
> 
> In case it runs on 2+ CPUs, it doesn't matter that
> one child is able to preempt parent, shared value
> is going to be increased regardless in all other
> children.

Acked, this test is broken on 2+ CPUs.

-- 
Cyril Hrubis
chrubis@suse.cz

------------------------------------------------------------------------------
Dive into the World of Parallel Programming The Go Parallel Website, sponsored
by Intel and developed in partnership with Slashdot Media, is your hub for all
things parallel software development, from weekly thought leadership blogs to
news, videos, case studies, tutorials and more. Take a look and join the 
conversation now. http://goparallel.sourceforge.net/
_______________________________________________
Ltp-list mailing list
Ltp-list@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/ltp-list

Re: [LTP] [PATCH 1/2] sched_setparam/9-1: fix hang on -rt kernel

[Cyril Hrubis]

Hi!
> As mentioned in [1], the testcase hangs on -rt kernel,
> because alarm(2) will never fire. The testcase runs with
> FIFO scheduling of higher priority than ksoftirqd.
> 
> This patch changes testcase in following way:
> 1. In good case, where child process preempts parent,
>    child is able to detect this condition and terminate.
> 2. In bad case, where preemption fails, testcase doesn't
>    rely on softirq (alarm), but hrtimer (time).
> 3. removes unnecessary sleep
> 
> [1] https://lkml.org/lkml/2006/1/10/481
> 
> Reported-by: Jiri Kastner <jkastner@redhat.com>
> Signed-off-by: Jan Stancek <jstancek@redhat.com>
> ---
>  .../conformance/interfaces/sched_setparam/9-1.c    | 48 ++++++++++++++--------
>  1 file changed, 32 insertions(+), 16 deletions(-)
> 
> diff --git a/testcases/open_posix_testsuite/conformance/interfaces/sched_setparam/9-1.c b/testcases/open_posix_testsuite/conformance/interfaces/sched_setparam/9-1.c
> index ed90cee..354e03e 100644
> --- a/testcases/open_posix_testsuite/conformance/interfaces/sched_setparam/9-1.c
> +++ b/testcases/open_posix_testsuite/conformance/interfaces/sched_setparam/9-1.c
> @@ -31,14 +31,15 @@
>   *      not, the test fail.
>   */
>  #define _GNU_SOURCE
> +#include <errno.h>
>  #include <sched.h>
>  #include <stdio.h>
>  #include <signal.h>
> +#include <stdlib.h>
>  #include <sys/ipc.h>
>  #include <sys/shm.h>
> +#include <time.h>
>  #include <unistd.h>
> -#include <stdlib.h>
> -#include <errno.h>
>  #include "posixtest.h"
>  #include "affinity.h"
>  
> @@ -55,6 +56,7 @@
>  
>  static int nb_cpu;
>  static int *shmptr;
> +static int mean_prio;
>  
>  static int get_ncpu(void)
>  {
> @@ -85,6 +87,7 @@ static int get_ncpu(void)
>  static void child_process(void)
>  {
>  	struct sched_param param;
> +	time_t t1, t2;
>  
>  	param.sched_priority = sched_get_priority_max(SCHED_FIFO);
>  	if (sched_setparam(getpid(), &param) != 0) {
> @@ -92,20 +95,33 @@ static void child_process(void)
>  		return;
>  	}
>  
> -	/* to avoid blocking */
> -	alarm(2);
> -	while (1) ;
> +	t1 = time(NULL);
> +	do {
> +		t2 = time(NULL);
> +	} while (difftime(t2, t1) <= 2);
>  }

Have you tested this code with dummy set_affinity()? Because on Linux
this doesn't get called at all.

>  static void test_process(void)
>  {
> -	/* to avoid blocking */
> -	alarm(2);
> -
> -	while (1) {
> -		(*shmptr)++;
> +	struct sched_param param;
> +	time_t t1, t2;
> +
> +	t1 = time(NULL);
> +	do {
> +		sched_getparam(getpid(), &param);
> +		(*shmptr) = param.sched_priority;
> +		/* if we can see that our priority has changed
> +		 * that means we preempted parent, so we are done */
> +		if ((*shmptr) == mean_prio)
> +			break;
> +
> +		t2 = time(NULL);
> +		/* immediately after parent forks us, we has same
> +		 * priority and compete with parent for same CPU,
> +		 * give parent chance to run and boost our priority */
>  		sched_yield();

Hmm, is this the case? Because the main thread should continue to run
until blocked and without the sleep(1) in the main() there is nothing
that could block the parent process between the fork and the priority
boost. Or am I mistaken?

> -	}
> +	} while (difftime(t2, t1) <= 2);
> +	exit(0);
>  }
>  
>  static void kill_children(int *child_pid)
> @@ -133,6 +149,8 @@ int main(void)
>  		nb_cpu = 1;
>  	}
>  
> +	mean_prio = (sched_get_priority_min(SCHED_FIFO) +
> +		sched_get_priority_max(SCHED_FIFO)) / 2;
>  	child_pid = malloc(nb_cpu * sizeof(int));
>  
>  	key = ftok("conformance/interfaces/sched_setparam/9-1.c", 1234);
> @@ -194,11 +212,9 @@ int main(void)
>  		return PTS_UNRESOLVED;
>  	}
>  
> -	sleep(1);
> -
> -	param.sched_priority = (sched_get_priority_min(SCHED_FIFO) +
> -				sched_get_priority_max(SCHED_FIFO)) / 2;
> -
> +	/* parent runs, which means test_process() gave up cpu,
> +	 * boost its priority and check it preempted parent */
> +	param.sched_priority = mean_prio;
>  	oldcount = *shmptr;

Shouldn't we rather initialize the shmptr to some known value (!=
mean_prio) now?

>  	if (sched_setparam(child_pid[i], &param) != 0) {
>  		perror("An error occurs when calling sched_setparam()");
> -- 
> 1.8.3.1
> 
> 
> ------------------------------------------------------------------------------
> Dive into the World of Parallel Programming The Go Parallel Website, sponsored
> by Intel and developed in partnership with Slashdot Media, is your hub for all
> things parallel software development, from weekly thought leadership blogs to
> news, videos, case studies, tutorials and more. Take a look and join the 
> conversation now. http://goparallel.sourceforge.net/
> _______________________________________________
> Ltp-list mailing list
> Ltp-list@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/ltp-list

-- 
Cyril Hrubis
chrubis@suse.cz

------------------------------------------------------------------------------
Dive into the World of Parallel Programming The Go Parallel Website, sponsored
by Intel and developed in partnership with Slashdot Media, is your hub for all
things parallel software development, from weekly thought leadership blogs to
news, videos, case studies, tutorials and more. Take a look and join the 
conversation now. http://goparallel.sourceforge.net/
_______________________________________________
Ltp-list mailing list
Ltp-list@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/ltp-list

Re: [LTP] [PATCH 1/2] sched_setparam/9-1: fix hang on -rt kernel

[Cyril Hrubis]

Hi!
> > >  static void test_process(void)
> > >  {
> > > -	/* to avoid blocking */
> > > -	alarm(2);
> > > -
> > > -	while (1) {
> > > -		(*shmptr)++;
> > > +	struct sched_param param;
> > > +	time_t t1, t2;
> > > +
> > > +	t1 = time(NULL);
> > > +	do {
> > > +		sched_getparam(getpid(), &param);
> > > +		(*shmptr) = param.sched_priority;
> > > +		/* if we can see that our priority has changed
> > > +		 * that means we preempted parent, so we are done */
> > > +		if ((*shmptr) == mean_prio)
> > > +			break;
> > > +
> > > +		t2 = time(NULL);
> > > +		/* immediately after parent forks us, we has same
> > > +		 * priority and compete with parent for same CPU,
> > > +		 * give parent chance to run and boost our priority */
> > >  		sched_yield();
> > 
> > Hmm, is this the case? Because the main thread should continue to run
> > until blocked and without the sleep(1) in the main() there is nothing
> > that could block the parent process between the fork and the priority
> > boost. Or am I mistaken?
> 
> I think you're right in scenario with single CPU.
> In theory, is it possible that set_affinity() would be dummy, and one of
> ncpu-1 children running with max FIFO priority would preempt parent
> allowing test_process() to run before its priority is changed?

Aren't the child proceesses forked with the same priority as the parent?
In this case they shouldn't get executed for the same reason unless they
are migrated other CPUs. But yes then it's a kind of unpredictable which
is the reason we have the affinity call there.

> I'd prefer to keep sched_yield() in test_process() as it poses
> no harm even in case parent can't be blocked/preempted.

Ok.

> > Shouldn't we rather initialize the shmptr to some known value (!=
> > mean_prio) now?
> 
> It's initialized to 0, after call to shmat.

Missed that, you are right.

-- 
Cyril Hrubis
chrubis@suse.cz

------------------------------------------------------------------------------
Dive into the World of Parallel Programming The Go Parallel Website, sponsored
by Intel and developed in partnership with Slashdot Media, is your hub for all
things parallel software development, from weekly thought leadership blogs to
news, videos, case studies, tutorials and more. Take a look and join the 
conversation now. http://goparallel.sourceforge.net/
_______________________________________________
Ltp-list mailing list
Ltp-list@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/ltp-list

Re: [LTP] [PATCH 1/2] sched_setparam/9-1: fix hang on -rt kernel

[Cyril Hrubis]

Hi!
> > > I think you're right in scenario with single CPU.
> > > In theory, is it possible that set_affinity() would be dummy, and one of
> > > ncpu-1 children running with max FIFO priority would preempt parent
> > > allowing test_process() to run before its priority is changed?
> > 
> > Aren't the child proceesses forked with the same priority as the parent?
> > In this case they shouldn't get executed for the same reason unless they
> > are migrated other CPUs. But yes then it's a kind of unpredictable which
> > is the reason we have the affinity call there.
> 
> Would you prefer I rewrite that comment? Or are you OK with the patch as it is?

It would be better to make the comment more clear.

-- 
Cyril Hrubis
chrubis@suse.cz

------------------------------------------------------------------------------
Dive into the World of Parallel Programming The Go Parallel Website, sponsored
by Intel and developed in partnership with Slashdot Media, is your hub for all
things parallel software development, from weekly thought leadership blogs to
news, videos, case studies, tutorials and more. Take a look and join the 
conversation now. http://goparallel.sourceforge.net/
_______________________________________________
Ltp-list mailing list
Ltp-list@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/ltp-list