[PATCH 0/3] selftests: timers: Convert some more of the timers tests to KTAP

[PATCH 0/3] selftests: timers: Convert some more of the timers tests to KTAP

[Mark Brown]

KTAP is the standard output format for selftests, providing a method for
systems running the selftests to get results from individual tests
rather than just a pass/fail for the test program as a whole.  While
many of the timers tests use KTAP some have custom output formats, let's
convert a few more to KTAP to make them work better in automation.

The posix_timers test made use of perror(), I've added a generic helper
to kselftest.h for that since it seems like it'll be useful elsewhere.

There are more tests that don't use KTAP, several of them just run a
single test so don't really benefit from KTAP and there were a couple
where the conversion was a bit more complex so I've left them for now.

Signed-off-by: Mark Brown <broonie@kernel.org>
---
Mark Brown (3):
      kselftest: Add a ksft_perror() helper
      selftests: timers: Convert posix_timers test to generate KTAP output
      selftests: timers: Convert nsleep-lat test to generate KTAP output

 tools/testing/selftests/kselftest.h           | 14 +++++
 tools/testing/selftests/timers/nsleep-lat.c   | 26 ++++-----
 tools/testing/selftests/timers/posix_timers.c | 81 ++++++++++++++-------------
 3 files changed, 67 insertions(+), 54 deletions(-)
---
base-commit: 6465e260f48790807eef06b583b38ca9789b6072
change-id: 20230926-ktap-posix-timers-67e978466185

Best regards,
-- 
Mark Brown <broonie@kernel.org>

[PATCH 1/3] kselftest: Add a ksft_perror() helper

[Mark Brown]

The standard library perror() function provides a convenient way to print
an error message based on the current errno but this doesn't play nicely
with KTAP output. Provide a helper which does an equivalent thing in a KTAP
compatible format.

nolibc doesn't have a strerror() and adding the table of strings required
doesn't seem like a good fit for what it's trying to do so when we're using
that only print the errno.

Signed-off-by: Mark Brown <broonie@kernel.org>
---
 tools/testing/selftests/kselftest.h | 14 ++++++++++++++
 1 file changed, 14 insertions(+)

diff --git a/tools/testing/selftests/kselftest.h b/tools/testing/selftests/kselftest.h
index 529d29a35900..af9f1202d423 100644
--- a/tools/testing/selftests/kselftest.h
+++ b/tools/testing/selftests/kselftest.h
@@ -48,6 +48,7 @@
 #include <stdlib.h>
 #include <unistd.h>
 #include <stdarg.h>
+#include <string.h>
 #include <stdio.h>
 #endif
 
@@ -155,6 +156,19 @@ static inline void ksft_print_msg(const char *msg, ...)
 	va_end(args);
 }
 
+static inline void ksft_perror(const char *msg)
+{
+#ifndef NOLIBC
+	ksft_print_msg("%s: %s (%d)\n", msg, strerror(errno), errno);
+#else
+	/*
+	 * nolibc doesn't provide strerror() and it seems
+	 * inappropriate to add one, just print the errno.
+	 */
+	ksft_print_msg("%s: %d)\n", msg, errno);
+#endif
+}
+
 static inline void ksft_test_result_pass(const char *msg, ...)
 {
 	int saved_errno = errno;

-- 
2.39.2

[PATCH 2/3] selftests: timers: Convert posix_timers test to generate KTAP output

[Mark Brown]

Currently the posix_timers test does not produce KTAP output but rather a
custom format. This means that we only get a pass/fail for the suite, not
for each individual test that the suite does. Convert to using the standard
kselftest output functions which result in KTAP output being generated.

As part of this fix the printing of diagnostics in the unlikely event that
the pthread APIs fail, these were using perror() but the API functions
directly return an error code instead of setting errno.

Signed-off-by: Mark Brown <broonie@kernel.org>
---
 tools/testing/selftests/timers/posix_timers.c | 81 ++++++++++++++-------------
 1 file changed, 41 insertions(+), 40 deletions(-)

diff --git a/tools/testing/selftests/timers/posix_timers.c b/tools/testing/selftests/timers/posix_timers.c
index 8a17c0e8d82b..d49dd3ffd0d9 100644
--- a/tools/testing/selftests/timers/posix_timers.c
+++ b/tools/testing/selftests/timers/posix_timers.c
@@ -76,22 +76,21 @@ static int check_diff(struct timeval start, struct timeval end)
 
 static int check_itimer(int which)
 {
+	const char *name;
 	int err;
 	struct timeval start, end;
 	struct itimerval val = {
 		.it_value.tv_sec = DELAY,
 	};
 
-	printf("Check itimer ");
-
 	if (which == ITIMER_VIRTUAL)
-		printf("virtual... ");
+		name = "ITIMER_VIRTUAL";
 	else if (which == ITIMER_PROF)
-		printf("prof... ");
+		name = "ITIMER_PROF";
 	else if (which == ITIMER_REAL)
-		printf("real... ");
-
-	fflush(stdout);
+		name = "ITIMER_REAL";
+	else
+		return -1;
 
 	done = 0;
 
@@ -104,13 +103,13 @@ static int check_itimer(int which)
 
 	err = gettimeofday(&start, NULL);
 	if (err < 0) {
-		perror("Can't call gettimeofday()\n");
+		ksft_perror("Can't call gettimeofday()");
 		return -1;
 	}
 
 	err = setitimer(which, &val, NULL);
 	if (err < 0) {
-		perror("Can't set timer\n");
+		ksft_perror("Can't set timer");
 		return -1;
 	}
 
@@ -123,20 +122,18 @@ static int check_itimer(int which)
 
 	err = gettimeofday(&end, NULL);
 	if (err < 0) {
-		perror("Can't call gettimeofday()\n");
+		ksft_perror("Can't call gettimeofday()");
 		return -1;
 	}
 
-	if (!check_diff(start, end))
-		printf("[OK]\n");
-	else
-		printf("[FAIL]\n");
+	ksft_test_result(check_diff(start, end) == 0, "%s\n", name);
 
 	return 0;
 }
 
 static int check_timer_create(int which)
 {
+	const char *type;
 	int err;
 	timer_t id;
 	struct timeval start, end;
@@ -144,31 +141,32 @@ static int check_timer_create(int which)
 		.it_value.tv_sec = DELAY,
 	};
 
-	printf("Check timer_create() ");
 	if (which == CLOCK_THREAD_CPUTIME_ID) {
-		printf("per thread... ");
+		type = "thread";
 	} else if (which == CLOCK_PROCESS_CPUTIME_ID) {
-		printf("per process... ");
+		type = "process";
+	} else {
+		ksft_print_msg("Unknown timer_create() type %d\n", which);
+		return -1;
 	}
-	fflush(stdout);
 
 	done = 0;
 	err = timer_create(which, NULL, &id);
 	if (err < 0) {
-		perror("Can't create timer\n");
+		ksft_perror("Can't create timer");
 		return -1;
 	}
 	signal(SIGALRM, sig_handler);
 
 	err = gettimeofday(&start, NULL);
 	if (err < 0) {
-		perror("Can't call gettimeofday()\n");
+		ksft_perror("Can't call gettimeofday()");
 		return -1;
 	}
 
 	err = timer_settime(id, 0, &val, NULL);
 	if (err < 0) {
-		perror("Can't set timer\n");
+		ksft_perror("Can't set timer");
 		return -1;
 	}
 
@@ -176,14 +174,12 @@ static int check_timer_create(int which)
 
 	err = gettimeofday(&end, NULL);
 	if (err < 0) {
-		perror("Can't call gettimeofday()\n");
+		ksft_perror("Can't call gettimeofday()");
 		return -1;
 	}
 
-	if (!check_diff(start, end))
-		printf("[OK]\n");
-	else
-		printf("[FAIL]\n");
+	ksft_test_result(check_diff(start, end) == 0,
+			 "timer_create() per %s\n", type);
 
 	return 0;
 }
@@ -220,25 +216,25 @@ static int check_timer_distribution(void)
 		.it_interval.tv_nsec = 1000 * 1000,
 	};
 
-	printf("Check timer_create() per process signal distribution... ");
-	fflush(stdout);
-
 	remain = nthreads + 1;  /* worker threads + this thread */
 	signal(SIGALRM, distribution_handler);
 	err = timer_create(CLOCK_PROCESS_CPUTIME_ID, NULL, &id);
 	if (err < 0) {
-		perror("Can't create timer\n");
+		ksft_perror("Can't create timer");
 		return -1;
 	}
 	err = timer_settime(id, 0, &val, NULL);
 	if (err < 0) {
-		perror("Can't set timer\n");
+		ksft_perror("Can't set timer");
 		return -1;
 	}
 
 	for (i = 0; i < nthreads; i++) {
-		if (pthread_create(&threads[i], NULL, distribution_thread, NULL)) {
-			perror("Can't create thread\n");
+		err = pthread_create(&threads[i], NULL, distribution_thread,
+				     NULL);
+		if (err) {
+			ksft_print_msg("Can't create thread: %s (%d)\n",
+				       strerror(errno), errno);
 			return -1;
 		}
 	}
@@ -247,25 +243,30 @@ static int check_timer_distribution(void)
 	while (__atomic_load_n(&remain, __ATOMIC_RELAXED));
 
 	for (i = 0; i < nthreads; i++) {
-		if (pthread_join(threads[i], NULL)) {
-			perror("Can't join thread\n");
+		err = pthread_join(threads[i], NULL);
+		if (err) {
+			ksft_print_msg("Can't join thread: %s (%d)\n",
+				       strerror(errno), errno);
 			return -1;
 		}
 	}
 
 	if (timer_delete(id)) {
-		perror("Can't delete timer\n");
+		ksft_perror("Can't delete timer");
 		return -1;
 	}
 
-	printf("[OK]\n");
+	ksft_test_result_pass("check_timer_distribution\n");
 	return 0;
 }
 
 int main(int argc, char **argv)
 {
-	printf("Testing posix timers. False negative may happen on CPU execution \n");
-	printf("based timers if other threads run on the CPU...\n");
+	ksft_print_header();
+	ksft_set_plan(6);
+
+	ksft_print_msg("Testing posix timers. False negative may happen on CPU execution \n");
+	ksft_print_msg("based timers if other threads run on the CPU...\n");
 
 	if (check_itimer(ITIMER_VIRTUAL) < 0)
 		return ksft_exit_fail();
@@ -294,5 +295,5 @@ int main(int argc, char **argv)
 	if (check_timer_distribution() < 0)
 		return ksft_exit_fail();
 
-	return ksft_exit_pass();
+	ksft_finished();
 }

-- 
2.39.2

[PATCH 3/3] selftests: timers: Convert nsleep-lat test to generate KTAP output

[Mark Brown]

Currently the nsleep-lat test does not produce KTAP output but rather a
custom format. This means that we only get a pass/fail for the suite, not
for each individual test that the suite does. Convert to using the standard
kselftest output functions which result in KTAP output being generated.

Signed-off-by: Mark Brown <broonie@kernel.org>
---
 tools/testing/selftests/timers/nsleep-lat.c | 26 ++++++++++++--------------
 1 file changed, 12 insertions(+), 14 deletions(-)

diff --git a/tools/testing/selftests/timers/nsleep-lat.c b/tools/testing/selftests/timers/nsleep-lat.c
index eb3e79ed7b4a..edb5acacf214 100644
--- a/tools/testing/selftests/timers/nsleep-lat.c
+++ b/tools/testing/selftests/timers/nsleep-lat.c
@@ -118,7 +118,7 @@ int nanosleep_lat_test(int clockid, long long ns)
 	clock_gettime(clockid, &end);
 
 	if (((timespec_sub(start, end)/count)-ns) > UNRESONABLE_LATENCY) {
-		printf("Large rel latency: %lld ns :", (timespec_sub(start, end)/count)-ns);
+		ksft_print_msg("Large rel latency: %lld ns :", (timespec_sub(start, end)/count)-ns);
 		return -1;
 	}
 
@@ -132,20 +132,23 @@ int nanosleep_lat_test(int clockid, long long ns)
 	}
 
 	if (latency/count > UNRESONABLE_LATENCY) {
-		printf("Large abs latency: %lld ns :", latency/count);
+		ksft_print_msg("Large abs latency: %lld ns :", latency/count);
 		return -1;
 	}
 
 	return 0;
 }
 
-
+#define SKIPPED_CLOCK_COUNT 3
 
 int main(int argc, char **argv)
 {
 	long long length;
 	int clockid, ret;
 
+	ksft_print_header();
+	ksft_set_plan(NR_CLOCKIDS - CLOCK_REALTIME - SKIPPED_CLOCK_COUNT);
+
 	for (clockid = CLOCK_REALTIME; clockid < NR_CLOCKIDS; clockid++) {
 
 		/* Skip cputime clockids since nanosleep won't increment cputime */
@@ -154,9 +157,6 @@ int main(int argc, char **argv)
 				clockid == CLOCK_HWSPECIFIC)
 			continue;
 
-		printf("nsleep latency %-26s ", clockstring(clockid));
-		fflush(stdout);
-
 		length = 10;
 		while (length <= (NSEC_PER_SEC * 10)) {
 			ret = nanosleep_lat_test(clockid, length);
@@ -167,14 +167,12 @@ int main(int argc, char **argv)
 		}
 
 		if (ret == UNSUPPORTED) {
-			printf("[UNSUPPORTED]\n");
-			continue;
-		}
-		if (ret < 0) {
-			printf("[FAILED]\n");
-			return ksft_exit_fail();
+			ksft_test_result_skip("%s\n", clockstring(clockid));
+		} else {
+			ksft_test_result(ret >= 0, "%s\n",
+					 clockstring(clockid));
 		}
-		printf("[OK]\n");
 	}
-	return ksft_exit_pass();
+
+	ksft_finished();
 }

-- 
2.39.2

Re: [PATCH 0/3] selftests: timers: Convert some more of the timers tests to KTAP

[Thomas Gleixner]

On Wed, Sep 27 2023 at 12:18, Mark Brown wrote:

> KTAP is the standard output format for selftests, providing a method for
> systems running the selftests to get results from individual tests
> rather than just a pass/fail for the test program as a whole.  While
> many of the timers tests use KTAP some have custom output formats, let's
> convert a few more to KTAP to make them work better in automation.
>
> The posix_timers test made use of perror(), I've added a generic helper
> to kselftest.h for that since it seems like it'll be useful elsewhere.
>
> There are more tests that don't use KTAP, several of them just run a
> single test so don't really benefit from KTAP and there were a couple
> where the conversion was a bit more complex so I've left them for now.

Acked-by: Thomas Gleixner <tglx@linutronix.de>