* [ptest-runner 2/8] Add missing stdint.h
2017-09-29 2:09 [ptest-runner 1/8] Corrected return code interpreter Jiwei Sun
@ 2017-09-29 2:09 ` Jiwei Sun
2017-09-29 2:09 ` [ptest-runner 3/8] Fix linker order for libcheck Jiwei Sun
` (6 subsequent siblings)
7 siblings, 0 replies; 12+ messages in thread
From: Jiwei Sun @ 2017-09-29 2:09 UTC (permalink / raw)
To: yocto; +Cc: anibal.limon
$ make check
cc -std=gnu99 -pedantic -Wall -Werror -I . -g -c tests/ptest_list.c -o tests/ptest_list.o
In file included from tests/ptest_list.c:25:0:
tests/ptest_list.c: In function ‘test_length’:
tests/ptest_list.c:85:2: error: unknown type name ‘intmax_t’
ck_assert_int_eq(ptest_list_length(head), ptests_num);
^
tests/ptest_list.c:85:2: error: unknown type name ‘intmax_t’
ck_assert_int_eq(ptest_list_length(head), ptests_num);
Signed-off-by: Jiwei Sun <jiwei.sun@windriver.com>
---
tests/ptest_list.c | 1 +
1 file changed, 1 insertion(+)
diff --git a/tests/ptest_list.c b/tests/ptest_list.c
index 5c0036d..2641620 100644
--- a/tests/ptest_list.c
+++ b/tests/ptest_list.c
@@ -21,6 +21,7 @@
#include <string.h>
#include <stdlib.h>
+#include <stdint.h>
#include <stdio.h>
#include <check.h>
#include <errno.h>
--
1.8.3.1
^ permalink raw reply related [flat|nested] 12+ messages in thread* [ptest-runner 3/8] Fix linker order for libcheck
2017-09-29 2:09 [ptest-runner 1/8] Corrected return code interpreter Jiwei Sun
2017-09-29 2:09 ` [ptest-runner 2/8] Add missing stdint.h Jiwei Sun
@ 2017-09-29 2:09 ` Jiwei Sun
2017-09-29 2:09 ` [ptest-runner 4/8] Add version(-v) since we change return code ABI Jiwei Sun
` (5 subsequent siblings)
7 siblings, 0 replies; 12+ messages in thread
From: Jiwei Sun @ 2017-09-29 2:09 UTC (permalink / raw)
To: yocto; +Cc: anibal.limon
$ gcc -lm -lrt -lpthread tests/main.o tests/ptest_list.o tests/utils.o utils.o ptest_list.o -o ptest-runner-test -lcheck
/usr/local/lib/libcheck.a(check.c.o): In function `tcase_create':
/home/edavnys/check/src/check.c:145: undefined reference to `floor'
/home/edavnys/check/src/check.c:148: undefined reference to `floor'
//usr/local/lib/libcheck.a(check.c.o): In function `tcase_set_timeout':
/home/edavnys/check/src/check.c:332: undefined reference to `floor'
/home/edavnys/check/src/check.c:334: undefined reference to `floor'
//usr/local/lib/libcheck.a(check.c.o): In function `check_get_clockid':
/home/edavnys/check/src/check.c:616: undefined reference to `timer_create'
/home/edavnys/check/src/check.c:618: undefined reference to `timer_delete'
//usr/local/lib/libcheck.a(check_run.c.o): In function `tcase_run_tfun_fork':
/home/edavnys/check/src/check_run.c:507: undefined reference to `timer_create'
/home/edavnys/check/src/check_run.c:515: undefined reference to `timer_settime'
/home/edavnys/check/src/check_run.c:529: undefined reference to `timer_delete'
collect2: error: ld returned 1 exit status
Signed-off-by: Jiwei Sun <jiwei.sun@windriver.com>
---
Makefile | 5 ++---
1 file changed, 2 insertions(+), 3 deletions(-)
diff --git a/Makefile b/Makefile
index 434b89f..3faf2e7 100644
--- a/Makefile
+++ b/Makefile
@@ -21,8 +21,7 @@ EXECUTABLE=ptest-runner
TEST_SOURCES=tests/main.c tests/ptest_list.c tests/utils.c $(BASE_SOURCES)
TEST_OBJECTS=$(TEST_SOURCES:.c=.o)
TEST_EXECUTABLE=ptest-runner-test
-TEST_LDFLAGS=-lm -lrt -lpthread
-TEST_LIBSTATIC=-lcheck
+TEST_LIBSTATIC=-lcheck -lrt -lm -lpthread
TEST_DATA=$(shell echo `pwd`/tests/data)
@@ -34,7 +33,7 @@ $(EXECUTABLE): $(OBJECTS)
tests: $(TEST_SOURCES) $(TEST_EXECUTABLE)
$(TEST_EXECUTABLE): $(TEST_OBJECTS)
- $(CC) $(LDFLAGS) $(TEST_LDFLAGS) $(TEST_OBJECTS) -o $@ $(TEST_LIBSTATIC)
+ $(CC) $(LDFLAGS) $(TEST_OBJECTS) -o $@ $(TEST_LIBSTATIC)
check: $(TEST_EXECUTABLE)
./$(TEST_EXECUTABLE) -d $(TEST_DATA)
--
1.8.3.1
^ permalink raw reply related [flat|nested] 12+ messages in thread* [ptest-runner 4/8] Add version(-v) since we change return code ABI
2017-09-29 2:09 [ptest-runner 1/8] Corrected return code interpreter Jiwei Sun
2017-09-29 2:09 ` [ptest-runner 2/8] Add missing stdint.h Jiwei Sun
2017-09-29 2:09 ` [ptest-runner 3/8] Fix linker order for libcheck Jiwei Sun
@ 2017-09-29 2:09 ` Jiwei Sun
2017-09-29 2:09 ` [ptest-runner 5/8] Add parallelism to TC execution Jiwei Sun
` (4 subsequent siblings)
7 siblings, 0 replies; 12+ messages in thread
From: Jiwei Sun @ 2017-09-29 2:09 UTC (permalink / raw)
To: yocto; +Cc: anibal.limon
Signed-off-by: Jiwei Sun <jiwei.sun@windriver.com>
---
main.c | 13 +++++++++++--
1 file changed, 11 insertions(+), 2 deletions(-)
diff --git a/main.c b/main.c
index 505829c..d524200 100644
--- a/main.c
+++ b/main.c
@@ -41,10 +41,16 @@
static inline void
print_usage(FILE *stream, char *progname)
{
- fprintf(stream, "Usage: %s [-d directory] [-l list] [-t timeout] [-x xml-filename]"
+ fprintf(stream, "Usage: %s [-v] [-d directory] [-l list] [-t timeout] [-x xml-filename]"
" [-h] [ptest1 ptest2 ...]\n", progname);
}
+static inline void
+print_version(FILE *stream, char *progname)
+{
+ fprintf(stream, "version: 3.0.0\n");
+}
+
int
main(int argc, char *argv[])
{
@@ -66,7 +72,7 @@ main(int argc, char *argv[])
opts.ptests = NULL;
opts.xml_filename = NULL;
- while ((opt = getopt(argc, argv, "d:lt:x:h")) != -1) {
+ while ((opt = getopt(argc, argv, "vd:lt:x:h")) != -1) {
switch (opt) {
case 'd':
free(opts.directory);
@@ -82,6 +88,9 @@ main(int argc, char *argv[])
case 'h':
print_usage(stdout, argv[0]);
exit(0);
+ case 'v':
+ print_version(stdout, argv[0]);
+ exit(0);
break;
case 'x':
free(opts.xml_filename);
--
1.8.3.1
^ permalink raw reply related [flat|nested] 12+ messages in thread* [ptest-runner 5/8] Add parallelism to TC execution
2017-09-29 2:09 [ptest-runner 1/8] Corrected return code interpreter Jiwei Sun
` (2 preceding siblings ...)
2017-09-29 2:09 ` [ptest-runner 4/8] Add version(-v) since we change return code ABI Jiwei Sun
@ 2017-09-29 2:09 ` Jiwei Sun
2017-10-03 15:31 ` Joshua Lock
2017-09-29 2:09 ` [ptest-runner 6/8] Add <system-out></system-out> to XML when tests fail Jiwei Sun
` (3 subsequent siblings)
7 siblings, 1 reply; 12+ messages in thread
From: Jiwei Sun @ 2017-09-29 2:09 UTC (permalink / raw)
To: yocto; +Cc: anibal.limon
Kinda only makes sense with JUnit test output.
Signed-off-by: Jiwei Sun <jiwei.sun@windriver.com>
---
ptest_list.c | 16 +--
ptest_list.h | 14 ++-
tests/data/glibc/ptest/run-pptest | 3 +
tests/data/parallel/ptest/run-pptest | 3 +
tests/ptest_list.c | 10 +-
tests/utils.c | 13 ++-
utils.c | 183 +++++++++++++++++++++++++++++++----
utils.h | 2 +-
8 files changed, 207 insertions(+), 37 deletions(-)
create mode 100755 tests/data/glibc/ptest/run-pptest
create mode 100755 tests/data/parallel/ptest/run-pptest
diff --git a/ptest_list.c b/ptest_list.c
index 2e1aa30..8a59383 100644
--- a/ptest_list.c
+++ b/ptest_list.c
@@ -27,8 +27,8 @@
#include "utils.h"
#include "ptest_list.h"
-#define VALIDATE_PTR_RINT(ptr) if (ptr == NULL) { errno = EINVAL; return -1; }
-#define VALIDATE_PTR_RNULL(ptr) if (ptr == NULL) { errno = EINVAL; return NULL; }
+#define VALIDATE_PTR_RINT(ptr) if (ptr == NULL) { errno = EINVAL; return -1; }
+#define VALIDATE_PTR_RNULL(ptr) if (ptr == NULL) { errno = EINVAL; return NULL; }
struct ptest_list *
ptest_list_alloc()
@@ -98,7 +98,7 @@ ptest_list_search(struct ptest_list *head, char *ptest)
VALIDATE_PTR_RNULL(ptest);
for (p = head; p != NULL; p = p->next) {
- if (p->ptest == NULL)
+ if (p->ptest == NULL)
continue;
if (strcmp(p->ptest, ptest) == 0) {
@@ -111,9 +111,12 @@ ptest_list_search(struct ptest_list *head, char *ptest)
}
struct ptest_list *
-ptest_list_add(struct ptest_list *head, char *ptest, char *run_ptest)
+ptest_list_add(struct ptest_list *head,
+ char *ptest,
+ char *run_ptest,
+ int parallel)
{
- struct ptest_list *n, *p;
+ struct ptest_list *n, *p;
VALIDATE_PTR_RNULL(head);
VALIDATE_PTR_RNULL(ptest);
@@ -124,6 +127,7 @@ ptest_list_add(struct ptest_list *head, char *ptest, char *run_ptest)
n->ptest = ptest;
n->run_ptest = run_ptest;
+ n->parallel = parallel;
n->prev = NULL;
n->next = NULL;
@@ -139,7 +143,7 @@ ptest_list_add(struct ptest_list *head, char *ptest, char *run_ptest)
struct ptest_list *
ptest_list_remove(struct ptest_list *head, char *ptest, int free)
{
- struct ptest_list *p;
+ struct ptest_list *p;
struct ptest_list *q, *r;
VALIDATE_PTR_RNULL(head);
diff --git a/ptest_list.h b/ptest_list.h
index 8b39485..e627921 100644
--- a/ptest_list.h
+++ b/ptest_list.h
@@ -25,12 +25,19 @@
#define PTEST_LIST_FREE_CLEAN(x) { ptest_list_free(x); x = NULL; }
#define PTEST_LIST_FREE_ALL_CLEAN(x) { ptest_list_free_all(x); x = NULL; }
-#define PTEST_LIST_ITERATE_START(head, p) for (p = head->next; p != NULL; p = p->next) {
+#define PTEST_LIST_ITERATE_START(head, p) for (p = head->next; p != NULL; p = p->next) {
#define PTEST_LIST_ITERATE_END }
+struct ptest_entry {
+ char *ptest;
+ char *run_ptest;
+ int parallel;
+};
+
struct ptest_list {
char *ptest;
char *run_ptest;
+ int parallel;
struct ptest_list *next;
struct ptest_list *prev;
@@ -42,7 +49,10 @@ extern int ptest_list_free_all(struct ptest_list *);
extern int ptest_list_length(struct ptest_list *);
extern struct ptest_list *ptest_list_search(struct ptest_list *, char *);
-extern struct ptest_list *ptest_list_add(struct ptest_list *, char *, char *);
+extern struct ptest_list *ptest_list_add(struct ptest_list *,
+ char *,
+ char *,
+ int);
extern struct ptest_list *ptest_list_remove(struct ptest_list *, char *, int);
#endif // _PTEST_RUNNER_LIST_H_
diff --git a/tests/data/glibc/ptest/run-pptest b/tests/data/glibc/ptest/run-pptest
new file mode 100755
index 0000000..f7f0316
--- /dev/null
+++ b/tests/data/glibc/ptest/run-pptest
@@ -0,0 +1,3 @@
+#!/bin/sh
+
+echo "glibc"
diff --git a/tests/data/parallel/ptest/run-pptest b/tests/data/parallel/ptest/run-pptest
new file mode 100755
index 0000000..23a0fff
--- /dev/null
+++ b/tests/data/parallel/ptest/run-pptest
@@ -0,0 +1,3 @@
+#!/bin/sh
+
+echo "parallel"
diff --git a/tests/ptest_list.c b/tests/ptest_list.c
index 2641620..a02af0e 100644
--- a/tests/ptest_list.c
+++ b/tests/ptest_list.c
@@ -49,7 +49,7 @@ END_TEST
START_TEST(test_add)
{
struct ptest_list *head = ptest_list_alloc();
- ck_assert(ptest_list_add(head, strdup("perl"), NULL) != NULL);
+ ck_assert(ptest_list_add(head, strdup("perl"), NULL, 0) != NULL);
ptest_list_free_all(head);
}
END_TEST
@@ -64,7 +64,7 @@ START_TEST(test_free_all)
head = ptest_list_alloc();
for (i = 0; i < ptests_num; i++)
- ptest_list_add(head, strdup(ptest_names[i]), NULL);
+ ptest_list_add(head, strdup(ptest_names[i]), NULL, 0);
ptest_list_free_all(head);
}
@@ -81,7 +81,7 @@ START_TEST(test_length)
head = ptest_list_alloc();
for (i = 0; i < ptests_num; i++)
- ptest_list_add(head, strdup(ptest_names[i]), NULL);
+ ptest_list_add(head, strdup(ptest_names[i]), NULL, 0);
ck_assert_int_eq(ptest_list_length(head), ptests_num);
ptest_list_free_all(head);
@@ -100,7 +100,7 @@ START_TEST(test_search)
head = ptest_list_alloc();
for (i = 0; i < ptests_num; i++) {
ptest = strdup(ptest_names[i]);
- ptest_list_add(head, ptest, NULL);
+ ptest_list_add(head, ptest, NULL, 0);
}
for (i = ptests_num - 1; i >= 0; i--)
@@ -119,7 +119,7 @@ START_TEST(test_remove)
for (i = 0; i < ptests_num; i++) {
ptest = strdup(ptest_names[i]);
- ptest_list_add(head, ptest, NULL);
+ ptest_list_add(head, ptest, NULL, 0);
}
/* Remove node free'ing */
diff --git a/tests/utils.c b/tests/utils.c
index ecf3e8a..9b1f442 100644
--- a/tests/utils.c
+++ b/tests/utils.c
@@ -40,11 +40,13 @@ static char *ptests_found[] = {
"fail",
"gcc",
"glibc",
+ "glibc",
"hang",
+ "parallel",
"python",
NULL
};
-static int ptests_found_length = 6;
+static int ptests_found_length = 8;
static char *ptests_not_found[] = {
"busybox",
"perl",
@@ -254,10 +256,15 @@ filecmp(FILE *fp1, FILE *fp2)
START_TEST(test_xml_pass)
FILE *xp;
+ struct ptest_entry entry;
+
xp = xml_create(2, "./test.xml");
ck_assert(xp != NULL);
- xml_add_case(xp, 0,"test1");
- xml_add_case(xp, 1,"test2");
+ entry.ptest = "test1";
+ entry.run_ptest = "run-ptest";
+ xml_add_case(xp, 0, &entry);
+ entry.ptest = "test2";
+ xml_add_case(xp, 1, &entry);
xml_finish(xp);
FILE *fp, *fr;
diff --git a/utils.c b/utils.c
index a07faec..056d7b7 100644
--- a/utils.c
+++ b/utils.c
@@ -42,7 +42,7 @@
#define GET_STIME_BUF_SIZE 1024
#define WAIT_CHILD_POLL_TIMEOUT_MS 200
-#define WAIT_CHILD_BUF_MAX_SIZE 1024
+#define WAIT_CHILD_BUF_MAX_SIZE 4096
static inline char *
get_stime(char *stime, size_t size)
@@ -108,7 +108,6 @@ get_available_ptests(const char *dir)
fail = 0;
for (i = 0; i < n; i++) {
char *run_ptest;
-
char *d_name = strdup(namelist[i]->d_name);
CHECK_ALLOCATION(d_name, sizeof(namelist[i]->d_name), 0);
if (d_name == NULL) {
@@ -124,7 +123,7 @@ get_available_ptests(const char *dir)
}
if (asprintf(&run_ptest, "%s/%s/ptest/run-ptest",
- dir, d_name) == -1) {
+ dir, d_name) == -1) {
fail = 1;
saved_errno = errno;
free(d_name);
@@ -133,18 +132,52 @@ get_available_ptests(const char *dir)
if (stat(run_ptest, &st_buf) == -1) {
free(run_ptest);
+ goto parallel;
+ }
+
+ if (!S_ISREG(st_buf.st_mode)) {
+ free(run_ptest);
+ goto parallel;
+ }
+
+ struct ptest_list *p = ptest_list_add(head,
+ d_name,
+ run_ptest,
+ 0);
+
+ CHECK_ALLOCATION(p, sizeof(struct ptest_list *), 0);
+ if (p == NULL) {
+ fail = 1;
+ saved_errno = errno;
+ free(run_ptest);
+ free(d_name);
+ break;
+ }
+
+ parallel:
+ if (asprintf(&run_ptest, "%s/%s/ptest/run-pptest",
+ dir, d_name) == -1) {
+ fail = 1;
+ saved_errno = errno;
free(d_name);
+ break;
+ }
+
+ if (stat(run_ptest, &st_buf) == -1) {
+ free(run_ptest);
continue;
}
if (!S_ISREG(st_buf.st_mode)) {
free(run_ptest);
- free(d_name);
continue;
}
- struct ptest_list *p = ptest_list_add(head,
- d_name, run_ptest);
+ p = ptest_list_add(head,
+ strdup(d_name),
+ run_ptest,
+ 1);
+
CHECK_ALLOCATION(p, sizeof(struct ptest_list *), 0);
if (p == NULL) {
fail = 1;
@@ -162,7 +195,7 @@ get_available_ptests(const char *dir)
if (fail) {
PTEST_LIST_FREE_ALL_CLEAN(head);
errno = saved_errno;
- break;
+ return NULL;
}
} while (0);
@@ -220,7 +253,10 @@ filter_ptests(struct ptest_list *head, char **ptests, int ptest_num)
break;
}
- if (ptest_list_add(head_new, ptest, run_ptest) == NULL) {
+ if (ptest_list_add(head_new,
+ ptest,
+ run_ptest,
+ n->parallel) == NULL) {
saved_errno = errno;
fail = 1;
break;
@@ -288,7 +324,7 @@ wait_child(const char *ptest_dir, const char *run_ptest, pid_t pid,
fwrite(buf, n, 1, fps[1]);
}
- sentinel = time(NULL);
+ sentinel = time(NULL);
} else if (timeout >= 0 && ((time(NULL) - sentinel) > timeout)) {
timeouted = 1;
kill(pid, SIGKILL);
@@ -303,11 +339,11 @@ wait_child(const char *ptest_dir, const char *run_ptest, pid_t pid,
if (timeouted)
fprintf(fps[0], "TIMEOUT: %s ", ptest_dir);
- if(WIFEXITED(status)) {
+ if(WIFEXITED(status) && pid > 0) {
fprintf(fps[0], "\nERROR: Exit status is %d\n", WEXITSTATUS(status));
return WEXITSTATUS(status);
}
- else if(WIFSIGNALED(status)) {
+ else if(WIFSIGNALED(status) && pid > 0) {
fprintf(fps[0], " Killed by signal\n");
return 127;
}
@@ -328,7 +364,7 @@ run_ptests(struct ptest_list *head, const struct ptest_options opts,
struct ptest_list *p;
char stime[GET_STIME_BUF_SIZE];
- pid_t child;
+ pid_t child, pid;
int pipefd_stdout[2];
int pipefd_stderr[2];
@@ -336,6 +372,7 @@ run_ptests(struct ptest_list *head, const struct ptest_options opts,
xh = xml_create(ptest_list_length(head), opts.xml_filename);
if (!xh)
exit(EXIT_FAILURE);
+ fflush(xh);
}
do
@@ -349,9 +386,90 @@ run_ptests(struct ptest_list *head, const struct ptest_options opts,
break;
}
+ fprintf(fp, "START PARALLEL: %s\n", progname);
+ PTEST_LIST_ITERATE_START(head, p);
+ char *ptest_dir;
+ int master;
+
+ if (!p->parallel)
+ continue;
+
+ master = fork();
+ if (master)
+ continue;
+
+ ptest_dir = strdup(p->run_ptest);
+ if (ptest_dir == NULL) {
+ rc = -1;
+ break;
+ }
+ dirname(ptest_dir);
+
+ child = fork();
+ if (child == -1) {
+ fprintf(fp, "ERROR: Fork %s\n", strerror(errno));
+ rc = -1;
+ break;
+ } else if (child == 0) {
+ run_child(p->run_ptest, pipefd_stdout[1], pipefd_stderr[1]);
+ } else {
+ int status;
+ int fds[2]; fds[0] = pipefd_stdout[0]; fds[1] = pipefd_stderr[0];
+ FILE *fps[2]; fps[0] = fp; fps[1] = fp_stderr;
+
+// fprintf(fp, "%s\n", get_stime(stime, GET_STIME_BUF_SIZE));
+ fprintf(fp, "BEGIN: %s\n", ptest_dir);
+
+ status = wait_child(ptest_dir, p->run_ptest, child,
+ opts.timeout, fds, fps);
+ if (status)
+ rc += 1;
+
+ if (opts.xml_filename)
+ xml_add_case(xh,
+ status,
+ (struct ptest_entry *)p);
+
+// fprintf(fp, "END: %s\n", ptest_dir);
+// fprintf(fp, "%s\n", get_stime(stime, GET_STIME_BUF_SIZE));
+ /* Let non-master gracefully terminate */
+ exit(0);
+ }
+ PTEST_LIST_ITERATE_END;
+
+ /* Reap all children before continuing */
+ while (1) {
+ int status;
+ pid = waitpid((pid_t)(-1), &status, WNOHANG);
+
+ /* Child reaped */
+ if (pid > 0) {
+ if (WIFEXITED(status)) {
+ if (WEXITSTATUS(status) == 0)
+ continue;
+ }
+ fprintf(fp, "One child failed to do its job pid:%u", pid);
+ exit(-1);
+ }
+
+ /* Still children alive */
+ if (pid == 0) {
+ usleep(100000); /* 100ms */
+ continue;
+ }
+
+ /* No more children to reap */
+ if (pid < 0)
+ break;
+ }
+
fprintf(fp, "START: %s\n", progname);
PTEST_LIST_ITERATE_START(head, p);
- char *ptest_dir = strdup(p->run_ptest);
+ char *ptest_dir;
+ if (p->parallel)
+ continue;
+
+ ptest_dir = strdup(p->run_ptest);
if (ptest_dir == NULL) {
rc = -1;
break;
@@ -379,7 +497,9 @@ run_ptests(struct ptest_list *head, const struct ptest_options opts,
rc += 1;
if (opts.xml_filename)
- xml_add_case(xh, status, ptest_dir);
+ xml_add_case(xh,
+ status,
+ (struct ptest_entry *)p);
fprintf(fp, "END: %s\n", ptest_dir);
fprintf(fp, "%s\n", get_stime(stime, GET_STIME_BUF_SIZE));
@@ -418,17 +538,40 @@ xml_create(int test_count, char *xml_filename)
}
void
-xml_add_case(FILE *xh, int status, const char *ptest_dir)
+xml_add_case(FILE *xh, int status, struct ptest_entry *ptest)
{
- fprintf(xh, "\t<testcase classname='%s' name='run-ptest'>\n", ptest_dir);
+ struct flock lock;
+ int fd;
+ char *bname;
+ fd = fileno(xh);
+ memset (&lock, 0, sizeof(lock));
+ lock.l_type = F_WRLCK;
+ fcntl (fd, F_SETLKW, &lock);
+ bname = strdup(ptest->run_ptest);
+
+ /* fprintf should guarantee atomicity for fprintfs within the same process */
if (status != 0) {
- fprintf(xh, "\t\t<failure type='exit_code'");
- fprintf(xh, " message='run-ptest exited with code: %d'>", status);
- fprintf(xh, "</failure>\n");
+ fprintf(xh, "\t<testcase classname='%s' name='%s'>\n" \
+ "\t\t<failure type='exit_code'" \
+ " message='run-ptest exited with code: %d'>" \
+ "</failure>\n" \
+ "\t</testcase>\n",
+ ptest->ptest,
+ basename(bname),
+ status);
+ }
+ else {
+ fprintf(xh, "\t<testcase classname='%s' name='%s'>\n" \
+ "\t</testcase>\n",
+ ptest->ptest,
+ basename(bname));
}
- fprintf(xh, "\t</testcase>\n");
+ fflush(xh);
+ lock.l_type = F_UNLCK;
+ fcntl (fd, F_SETLKW, &lock);
+ free(bname);
}
void
diff --git a/utils.h b/utils.h
index 8fa20a8..d0ef735 100644
--- a/utils.h
+++ b/utils.h
@@ -47,7 +47,7 @@ extern int run_ptests(struct ptest_list *, const struct ptest_options,
const char *, FILE *, FILE *);
extern FILE *xml_create(int, char *);
-extern void xml_add_case(FILE *, int, const char *);
+extern void xml_add_case(FILE *, int, struct ptest_entry *);
extern void xml_finish(FILE *);
#endif
--
1.8.3.1
^ permalink raw reply related [flat|nested] 12+ messages in thread* Re: [ptest-runner 5/8] Add parallelism to TC execution
2017-09-29 2:09 ` [ptest-runner 5/8] Add parallelism to TC execution Jiwei Sun
@ 2017-10-03 15:31 ` Joshua Lock
0 siblings, 0 replies; 12+ messages in thread
From: Joshua Lock @ 2017-10-03 15:31 UTC (permalink / raw)
To: Jiwei Sun, yocto@yoctoproject.org
Thanks for the patch, could you help potential reviewers to understand
what the patch adds and why by improving the commit message?
There's a lot of noise in this patch with unrelated whitespace changes
and formatting changes. Please re-send without those changes. Existing
coding style should be maintained.
Thanks,
Joshua
On 29/09/17 03:09, Jiwei Sun wrote:
> Kinda only makes sense with JUnit test output.
>
> Signed-off-by: Jiwei Sun <jiwei.sun@windriver.com>
> ---
> ptest_list.c | 16 +--
> ptest_list.h | 14 ++-
> tests/data/glibc/ptest/run-pptest | 3 +
> tests/data/parallel/ptest/run-pptest | 3 +
> tests/ptest_list.c | 10 +-
> tests/utils.c | 13 ++-
> utils.c | 183 +++++++++++++++++++++++++++++++----
> utils.h | 2 +-
> 8 files changed, 207 insertions(+), 37 deletions(-)
> create mode 100755 tests/data/glibc/ptest/run-pptest
> create mode 100755 tests/data/parallel/ptest/run-pptest
>
> diff --git a/ptest_list.c b/ptest_list.c
> index 2e1aa30..8a59383 100644
> --- a/ptest_list.c
> +++ b/ptest_list.c
> @@ -27,8 +27,8 @@
> #include "utils.h"
> #include "ptest_list.h"
>
> -#define VALIDATE_PTR_RINT(ptr) if (ptr == NULL) { errno = EINVAL; return -1; }
> -#define VALIDATE_PTR_RNULL(ptr) if (ptr == NULL) { errno = EINVAL; return NULL; }
> +#define VALIDATE_PTR_RINT(ptr) if (ptr == NULL) { errno = EINVAL; return -1; }
> +#define VALIDATE_PTR_RNULL(ptr) if (ptr == NULL) { errno = EINVAL; return NULL; }
>
> struct ptest_list *
> ptest_list_alloc()
> @@ -98,7 +98,7 @@ ptest_list_search(struct ptest_list *head, char *ptest)
> VALIDATE_PTR_RNULL(ptest);
>
> for (p = head; p != NULL; p = p->next) {
> - if (p->ptest == NULL)
> + if (p->ptest == NULL)
> continue;
>
> if (strcmp(p->ptest, ptest) == 0) {
> @@ -111,9 +111,12 @@ ptest_list_search(struct ptest_list *head, char *ptest)
> }
>
> struct ptest_list *
> -ptest_list_add(struct ptest_list *head, char *ptest, char *run_ptest)
> +ptest_list_add(struct ptest_list *head,
> + char *ptest,
> + char *run_ptest,
> + int parallel)
> {
> - struct ptest_list *n, *p;
> + struct ptest_list *n, *p;
>
> VALIDATE_PTR_RNULL(head);
> VALIDATE_PTR_RNULL(ptest);
> @@ -124,6 +127,7 @@ ptest_list_add(struct ptest_list *head, char *ptest, char *run_ptest)
>
> n->ptest = ptest;
> n->run_ptest = run_ptest;
> + n->parallel = parallel;
>
> n->prev = NULL;
> n->next = NULL;
> @@ -139,7 +143,7 @@ ptest_list_add(struct ptest_list *head, char *ptest, char *run_ptest)
> struct ptest_list *
> ptest_list_remove(struct ptest_list *head, char *ptest, int free)
> {
> - struct ptest_list *p;
> + struct ptest_list *p;
> struct ptest_list *q, *r;
>
> VALIDATE_PTR_RNULL(head);
> diff --git a/ptest_list.h b/ptest_list.h
> index 8b39485..e627921 100644
> --- a/ptest_list.h
> +++ b/ptest_list.h
> @@ -25,12 +25,19 @@
> #define PTEST_LIST_FREE_CLEAN(x) { ptest_list_free(x); x = NULL; }
> #define PTEST_LIST_FREE_ALL_CLEAN(x) { ptest_list_free_all(x); x = NULL; }
>
> -#define PTEST_LIST_ITERATE_START(head, p) for (p = head->next; p != NULL; p = p->next) {
> +#define PTEST_LIST_ITERATE_START(head, p) for (p = head->next; p != NULL; p = p->next) {
> #define PTEST_LIST_ITERATE_END }
>
> +struct ptest_entry {
> + char *ptest;
> + char *run_ptest;
> + int parallel;
> +};
> +
> struct ptest_list {
> char *ptest;
> char *run_ptest;
> + int parallel;
>
> struct ptest_list *next;
> struct ptest_list *prev;
> @@ -42,7 +49,10 @@ extern int ptest_list_free_all(struct ptest_list *);
>
> extern int ptest_list_length(struct ptest_list *);
> extern struct ptest_list *ptest_list_search(struct ptest_list *, char *);
> -extern struct ptest_list *ptest_list_add(struct ptest_list *, char *, char *);
> +extern struct ptest_list *ptest_list_add(struct ptest_list *,
> + char *,
> + char *,
> + int);
> extern struct ptest_list *ptest_list_remove(struct ptest_list *, char *, int);
>
> #endif // _PTEST_RUNNER_LIST_H_
> diff --git a/tests/data/glibc/ptest/run-pptest b/tests/data/glibc/ptest/run-pptest
> new file mode 100755
> index 0000000..f7f0316
> --- /dev/null
> +++ b/tests/data/glibc/ptest/run-pptest
> @@ -0,0 +1,3 @@
> +#!/bin/sh
> +
> +echo "glibc"
> diff --git a/tests/data/parallel/ptest/run-pptest b/tests/data/parallel/ptest/run-pptest
> new file mode 100755
> index 0000000..23a0fff
> --- /dev/null
> +++ b/tests/data/parallel/ptest/run-pptest
> @@ -0,0 +1,3 @@
> +#!/bin/sh
> +
> +echo "parallel"
> diff --git a/tests/ptest_list.c b/tests/ptest_list.c
> index 2641620..a02af0e 100644
> --- a/tests/ptest_list.c
> +++ b/tests/ptest_list.c
> @@ -49,7 +49,7 @@ END_TEST
> START_TEST(test_add)
> {
> struct ptest_list *head = ptest_list_alloc();
> - ck_assert(ptest_list_add(head, strdup("perl"), NULL) != NULL);
> + ck_assert(ptest_list_add(head, strdup("perl"), NULL, 0) != NULL);
> ptest_list_free_all(head);
> }
> END_TEST
> @@ -64,7 +64,7 @@ START_TEST(test_free_all)
>
> head = ptest_list_alloc();
> for (i = 0; i < ptests_num; i++)
> - ptest_list_add(head, strdup(ptest_names[i]), NULL);
> + ptest_list_add(head, strdup(ptest_names[i]), NULL, 0);
>
> ptest_list_free_all(head);
> }
> @@ -81,7 +81,7 @@ START_TEST(test_length)
>
> head = ptest_list_alloc();
> for (i = 0; i < ptests_num; i++)
> - ptest_list_add(head, strdup(ptest_names[i]), NULL);
> + ptest_list_add(head, strdup(ptest_names[i]), NULL, 0);
>
> ck_assert_int_eq(ptest_list_length(head), ptests_num);
> ptest_list_free_all(head);
> @@ -100,7 +100,7 @@ START_TEST(test_search)
> head = ptest_list_alloc();
> for (i = 0; i < ptests_num; i++) {
> ptest = strdup(ptest_names[i]);
> - ptest_list_add(head, ptest, NULL);
> + ptest_list_add(head, ptest, NULL, 0);
> }
>
> for (i = ptests_num - 1; i >= 0; i--)
> @@ -119,7 +119,7 @@ START_TEST(test_remove)
>
> for (i = 0; i < ptests_num; i++) {
> ptest = strdup(ptest_names[i]);
> - ptest_list_add(head, ptest, NULL);
> + ptest_list_add(head, ptest, NULL, 0);
> }
>
> /* Remove node free'ing */
> diff --git a/tests/utils.c b/tests/utils.c
> index ecf3e8a..9b1f442 100644
> --- a/tests/utils.c
> +++ b/tests/utils.c
> @@ -40,11 +40,13 @@ static char *ptests_found[] = {
> "fail",
> "gcc",
> "glibc",
> + "glibc",
> "hang",
> + "parallel",
> "python",
> NULL
> };
> -static int ptests_found_length = 6;
> +static int ptests_found_length = 8;
> static char *ptests_not_found[] = {
> "busybox",
> "perl",
> @@ -254,10 +256,15 @@ filecmp(FILE *fp1, FILE *fp2)
>
> START_TEST(test_xml_pass)
> FILE *xp;
> + struct ptest_entry entry;
> +
> xp = xml_create(2, "./test.xml");
> ck_assert(xp != NULL);
> - xml_add_case(xp, 0,"test1");
> - xml_add_case(xp, 1,"test2");
> + entry.ptest = "test1";
> + entry.run_ptest = "run-ptest";
> + xml_add_case(xp, 0, &entry);
> + entry.ptest = "test2";
> + xml_add_case(xp, 1, &entry);
> xml_finish(xp);
>
> FILE *fp, *fr;
> diff --git a/utils.c b/utils.c
> index a07faec..056d7b7 100644
> --- a/utils.c
> +++ b/utils.c
> @@ -42,7 +42,7 @@
>
> #define GET_STIME_BUF_SIZE 1024
> #define WAIT_CHILD_POLL_TIMEOUT_MS 200
> -#define WAIT_CHILD_BUF_MAX_SIZE 1024
> +#define WAIT_CHILD_BUF_MAX_SIZE 4096
>
> static inline char *
> get_stime(char *stime, size_t size)
> @@ -108,7 +108,6 @@ get_available_ptests(const char *dir)
> fail = 0;
> for (i = 0; i < n; i++) {
> char *run_ptest;
> -
> char *d_name = strdup(namelist[i]->d_name);
> CHECK_ALLOCATION(d_name, sizeof(namelist[i]->d_name), 0);
> if (d_name == NULL) {
> @@ -124,7 +123,7 @@ get_available_ptests(const char *dir)
> }
>
> if (asprintf(&run_ptest, "%s/%s/ptest/run-ptest",
> - dir, d_name) == -1) {
> + dir, d_name) == -1) {
> fail = 1;
> saved_errno = errno;
> free(d_name);
> @@ -133,18 +132,52 @@ get_available_ptests(const char *dir)
>
> if (stat(run_ptest, &st_buf) == -1) {
> free(run_ptest);
> + goto parallel;
> + }
> +
> + if (!S_ISREG(st_buf.st_mode)) {
> + free(run_ptest);
> + goto parallel;
> + }
> +
> + struct ptest_list *p = ptest_list_add(head,
> + d_name,
> + run_ptest,
> + 0);
> +
> + CHECK_ALLOCATION(p, sizeof(struct ptest_list *), 0);
> + if (p == NULL) {
> + fail = 1;
> + saved_errno = errno;
> + free(run_ptest);
> + free(d_name);
> + break;
> + }
> +
> + parallel:
> + if (asprintf(&run_ptest, "%s/%s/ptest/run-pptest",
> + dir, d_name) == -1) {
> + fail = 1;
> + saved_errno = errno;
> free(d_name);
> + break;
> + }
> +
> + if (stat(run_ptest, &st_buf) == -1) {
> + free(run_ptest);
> continue;
> }
>
> if (!S_ISREG(st_buf.st_mode)) {
> free(run_ptest);
> - free(d_name);
> continue;
> }
>
> - struct ptest_list *p = ptest_list_add(head,
> - d_name, run_ptest);
> + p = ptest_list_add(head,
> + strdup(d_name),
> + run_ptest,
> + 1);
> +
> CHECK_ALLOCATION(p, sizeof(struct ptest_list *), 0);
> if (p == NULL) {
> fail = 1;
> @@ -162,7 +195,7 @@ get_available_ptests(const char *dir)
> if (fail) {
> PTEST_LIST_FREE_ALL_CLEAN(head);
> errno = saved_errno;
> - break;
> + return NULL;
> }
> } while (0);
>
> @@ -220,7 +253,10 @@ filter_ptests(struct ptest_list *head, char **ptests, int ptest_num)
> break;
> }
>
> - if (ptest_list_add(head_new, ptest, run_ptest) == NULL) {
> + if (ptest_list_add(head_new,
> + ptest,
> + run_ptest,
> + n->parallel) == NULL) {
> saved_errno = errno;
> fail = 1;
> break;
> @@ -288,7 +324,7 @@ wait_child(const char *ptest_dir, const char *run_ptest, pid_t pid,
> fwrite(buf, n, 1, fps[1]);
> }
>
> - sentinel = time(NULL);
> + sentinel = time(NULL);
> } else if (timeout >= 0 && ((time(NULL) - sentinel) > timeout)) {
> timeouted = 1;
> kill(pid, SIGKILL);
> @@ -303,11 +339,11 @@ wait_child(const char *ptest_dir, const char *run_ptest, pid_t pid,
> if (timeouted)
> fprintf(fps[0], "TIMEOUT: %s ", ptest_dir);
>
> - if(WIFEXITED(status)) {
> + if(WIFEXITED(status) && pid > 0) {
> fprintf(fps[0], "\nERROR: Exit status is %d\n", WEXITSTATUS(status));
> return WEXITSTATUS(status);
> }
> - else if(WIFSIGNALED(status)) {
> + else if(WIFSIGNALED(status) && pid > 0) {
> fprintf(fps[0], " Killed by signal\n");
> return 127;
> }
> @@ -328,7 +364,7 @@ run_ptests(struct ptest_list *head, const struct ptest_options opts,
> struct ptest_list *p;
> char stime[GET_STIME_BUF_SIZE];
>
> - pid_t child;
> + pid_t child, pid;
> int pipefd_stdout[2];
> int pipefd_stderr[2];
>
> @@ -336,6 +372,7 @@ run_ptests(struct ptest_list *head, const struct ptest_options opts,
> xh = xml_create(ptest_list_length(head), opts.xml_filename);
> if (!xh)
> exit(EXIT_FAILURE);
> + fflush(xh);
> }
>
> do
> @@ -349,9 +386,90 @@ run_ptests(struct ptest_list *head, const struct ptest_options opts,
> break;
> }
>
> + fprintf(fp, "START PARALLEL: %s\n", progname);
> + PTEST_LIST_ITERATE_START(head, p);
> + char *ptest_dir;
> + int master;
> +
> + if (!p->parallel)
> + continue;
> +
> + master = fork();
> + if (master)
> + continue;
> +
> + ptest_dir = strdup(p->run_ptest);
> + if (ptest_dir == NULL) {
> + rc = -1;
> + break;
> + }
> + dirname(ptest_dir);
> +
> + child = fork();
> + if (child == -1) {
> + fprintf(fp, "ERROR: Fork %s\n", strerror(errno));
> + rc = -1;
> + break;
> + } else if (child == 0) {
> + run_child(p->run_ptest, pipefd_stdout[1], pipefd_stderr[1]);
> + } else {
> + int status;
> + int fds[2]; fds[0] = pipefd_stdout[0]; fds[1] = pipefd_stderr[0];
> + FILE *fps[2]; fps[0] = fp; fps[1] = fp_stderr;
> +
> +// fprintf(fp, "%s\n", get_stime(stime, GET_STIME_BUF_SIZE));
> + fprintf(fp, "BEGIN: %s\n", ptest_dir);
> +
> + status = wait_child(ptest_dir, p->run_ptest, child,
> + opts.timeout, fds, fps);
> + if (status)
> + rc += 1;
> +
> + if (opts.xml_filename)
> + xml_add_case(xh,
> + status,
> + (struct ptest_entry *)p);
> +
> +// fprintf(fp, "END: %s\n", ptest_dir);
> +// fprintf(fp, "%s\n", get_stime(stime, GET_STIME_BUF_SIZE));
> + /* Let non-master gracefully terminate */
> + exit(0);
> + }
> + PTEST_LIST_ITERATE_END;
> +
> + /* Reap all children before continuing */
> + while (1) {
> + int status;
> + pid = waitpid((pid_t)(-1), &status, WNOHANG);
> +
> + /* Child reaped */
> + if (pid > 0) {
> + if (WIFEXITED(status)) {
> + if (WEXITSTATUS(status) == 0)
> + continue;
> + }
> + fprintf(fp, "One child failed to do its job pid:%u", pid);
> + exit(-1);
> + }
> +
> + /* Still children alive */
> + if (pid == 0) {
> + usleep(100000); /* 100ms */
> + continue;
> + }
> +
> + /* No more children to reap */
> + if (pid < 0)
> + break;
> + }
> +
> fprintf(fp, "START: %s\n", progname);
> PTEST_LIST_ITERATE_START(head, p);
> - char *ptest_dir = strdup(p->run_ptest);
> + char *ptest_dir;
> + if (p->parallel)
> + continue;
> +
> + ptest_dir = strdup(p->run_ptest);
> if (ptest_dir == NULL) {
> rc = -1;
> break;
> @@ -379,7 +497,9 @@ run_ptests(struct ptest_list *head, const struct ptest_options opts,
> rc += 1;
>
> if (opts.xml_filename)
> - xml_add_case(xh, status, ptest_dir);
> + xml_add_case(xh,
> + status,
> + (struct ptest_entry *)p);
>
> fprintf(fp, "END: %s\n", ptest_dir);
> fprintf(fp, "%s\n", get_stime(stime, GET_STIME_BUF_SIZE));
> @@ -418,17 +538,40 @@ xml_create(int test_count, char *xml_filename)
> }
>
> void
> -xml_add_case(FILE *xh, int status, const char *ptest_dir)
> +xml_add_case(FILE *xh, int status, struct ptest_entry *ptest)
> {
> - fprintf(xh, "\t<testcase classname='%s' name='run-ptest'>\n", ptest_dir);
> + struct flock lock;
> + int fd;
> + char *bname;
>
> + fd = fileno(xh);
> + memset (&lock, 0, sizeof(lock));
> + lock.l_type = F_WRLCK;
> + fcntl (fd, F_SETLKW, &lock);
> + bname = strdup(ptest->run_ptest);
> +
> + /* fprintf should guarantee atomicity for fprintfs within the same process */
> if (status != 0) {
> - fprintf(xh, "\t\t<failure type='exit_code'");
> - fprintf(xh, " message='run-ptest exited with code: %d'>", status);
> - fprintf(xh, "</failure>\n");
> + fprintf(xh, "\t<testcase classname='%s' name='%s'>\n" \
> + "\t\t<failure type='exit_code'" \
> + " message='run-ptest exited with code: %d'>" \
> + "</failure>\n" \
> + "\t</testcase>\n",
> + ptest->ptest,
> + basename(bname),
> + status);
> + }
> + else {
> + fprintf(xh, "\t<testcase classname='%s' name='%s'>\n" \
> + "\t</testcase>\n",
> + ptest->ptest,
> + basename(bname));
> }
>
> - fprintf(xh, "\t</testcase>\n");
> + fflush(xh);
> + lock.l_type = F_UNLCK;
> + fcntl (fd, F_SETLKW, &lock);
> + free(bname);
> }
>
> void
> diff --git a/utils.h b/utils.h
> index 8fa20a8..d0ef735 100644
> --- a/utils.h
> +++ b/utils.h
> @@ -47,7 +47,7 @@ extern int run_ptests(struct ptest_list *, const struct ptest_options,
> const char *, FILE *, FILE *);
>
> extern FILE *xml_create(int, char *);
> -extern void xml_add_case(FILE *, int, const char *);
> +extern void xml_add_case(FILE *, int, struct ptest_entry *);
> extern void xml_finish(FILE *);
>
> #endif
>
^ permalink raw reply [flat|nested] 12+ messages in thread
* [ptest-runner 6/8] Add <system-out></system-out> to XML when tests fail
2017-09-29 2:09 [ptest-runner 1/8] Corrected return code interpreter Jiwei Sun
` (3 preceding siblings ...)
2017-09-29 2:09 ` [ptest-runner 5/8] Add parallelism to TC execution Jiwei Sun
@ 2017-09-29 2:09 ` Jiwei Sun
2017-10-03 15:31 ` Joshua Lock
2017-09-29 2:09 ` [ptest-runner 7/8] Introduce a ret-code for timeouts Jiwei Sun
` (2 subsequent siblings)
7 siblings, 1 reply; 12+ messages in thread
From: Jiwei Sun @ 2017-09-29 2:09 UTC (permalink / raw)
To: yocto; +Cc: anibal.limon
* Introduce a maximum number of concurrent ptests, since
we are dealing with embedded targets.
* Add stdout || stderr output from ptests into XML.
For testcases running in parallel, cluttering stdout won't
help anyone.
Signed-off-by: Jiwei Sun <jiwei.sun@windriver.com>
---
main.c | 2 +
tests/data/reference.xml | 1 +
tests/utils.c | 4 +-
utils.c | 99 +++++++++++++++++++++++++++++-------------------
utils.h | 3 +-
5 files changed, 67 insertions(+), 42 deletions(-)
diff --git a/main.c b/main.c
index d524200..a201e87 100644
--- a/main.c
+++ b/main.c
@@ -37,6 +37,7 @@
#define DEFAULT_DIRECTORY "/usr/lib"
#define DEFAULT_TIMEOUT 300
+#define DEFAULT_MAX_PARALLEL 64
static inline void
print_usage(FILE *stream, char *progname)
@@ -69,6 +70,7 @@ main(int argc, char *argv[])
opts.directory = strdup(DEFAULT_DIRECTORY);
opts.list = 0;
opts.timeout = DEFAULT_TIMEOUT;
+ opts.max_parallel = DEFAULT_MAX_PARALLEL;
opts.ptests = NULL;
opts.xml_filename = NULL;
diff --git a/tests/data/reference.xml b/tests/data/reference.xml
index 91522c7..a23b44b 100644
--- a/tests/data/reference.xml
+++ b/tests/data/reference.xml
@@ -4,5 +4,6 @@
</testcase>
<testcase classname='test2' name='run-ptest'>
<failure type='exit_code' message='run-ptest exited with code: 1'></failure>
+ <system-out>ERROR</system-out>
</testcase>
</testsuite>
diff --git a/tests/utils.c b/tests/utils.c
index 9b1f442..bb799f4 100644
--- a/tests/utils.c
+++ b/tests/utils.c
@@ -262,9 +262,9 @@ START_TEST(test_xml_pass)
ck_assert(xp != NULL);
entry.ptest = "test1";
entry.run_ptest = "run-ptest";
- xml_add_case(xp, 0, &entry);
+ xml_add_case(xp, 0, &entry, "");
entry.ptest = "test2";
- xml_add_case(xp, 1, &entry);
+ xml_add_case(xp, 1, &entry, "ERROR");
xml_finish(xp);
FILE *fp, *fr;
diff --git a/utils.c b/utils.c
index 056d7b7..0e97ac4 100644
--- a/utils.c
+++ b/utils.c
@@ -289,8 +289,13 @@ run_child(char *run_ptest, int fd_stdout, int fd_stderr)
}
static inline int
-wait_child(const char *ptest_dir, const char *run_ptest, pid_t pid,
- int timeout, int *fds, FILE **fps)
+wait_child(const char *ptest_dir,
+ const char *run_ptest,
+ pid_t pid,
+ int timeout,
+ int *fds,
+ FILE **fps,
+ char *buf)
{
struct pollfd pfds[2];
time_t sentinel;
@@ -311,7 +316,6 @@ wait_child(const char *ptest_dir, const char *run_ptest, pid_t pid,
r = poll(pfds, 2, WAIT_CHILD_POLL_TIMEOUT_MS);
if (r > 0) {
- char buf[WAIT_CHILD_BUF_MAX_SIZE];
ssize_t n;
if (pfds[0].revents != 0) {
@@ -354,6 +358,35 @@ wait_child(const char *ptest_dir, const char *run_ptest, pid_t pid,
return status;
}
+static void reap_all_children(FILE *fp) {
+ /* Reap all children before continuing */
+ while (1) {
+ int status;
+ pid_t pid;
+ pid = waitpid((pid_t)(-1), &status, WNOHANG);
+
+ /* Child reaped */
+ if (pid > 0) {
+ if (WIFEXITED(status)) {
+ if (WEXITSTATUS(status) == 0)
+ continue;
+ }
+ fprintf(fp, "One child failed to do its job pid:%u", pid);
+ exit(-1);
+ }
+
+ /* Still children alive */
+ if (pid == 0) {
+ usleep(100000); /* 100ms */
+ continue;
+ }
+
+ /* No more children to reap */
+ if (pid < 0)
+ break;
+ }
+}
+
int
run_ptests(struct ptest_list *head, const struct ptest_options opts,
const char *progname, FILE *fp, FILE *fp_stderr)
@@ -363,8 +396,8 @@ run_ptests(struct ptest_list *head, const struct ptest_options opts,
struct ptest_list *p;
char stime[GET_STIME_BUF_SIZE];
-
- pid_t child, pid;
+ int nr_children = 0;
+ pid_t child;
int pipefd_stdout[2];
int pipefd_stderr[2];
@@ -395,8 +428,13 @@ run_ptests(struct ptest_list *head, const struct ptest_options opts,
continue;
master = fork();
- if (master)
+ if (master) {
+ if (++nr_children >= opts.max_parallel) {
+ reap_all_children(fp);
+ nr_children = 0;
+ }
continue;
+ }
ptest_dir = strdup(p->run_ptest);
if (ptest_dir == NULL) {
@@ -416,53 +454,30 @@ run_ptests(struct ptest_list *head, const struct ptest_options opts,
int status;
int fds[2]; fds[0] = pipefd_stdout[0]; fds[1] = pipefd_stderr[0];
FILE *fps[2]; fps[0] = fp; fps[1] = fp_stderr;
+ char buf[WAIT_CHILD_BUF_MAX_SIZE];
// fprintf(fp, "%s\n", get_stime(stime, GET_STIME_BUF_SIZE));
fprintf(fp, "BEGIN: %s\n", ptest_dir);
status = wait_child(ptest_dir, p->run_ptest, child,
- opts.timeout, fds, fps);
+ opts.timeout, fds, fps, buf);
if (status)
rc += 1;
if (opts.xml_filename)
xml_add_case(xh,
status,
- (struct ptest_entry *)p);
+ (struct ptest_entry *)p,
+ buf);
-// fprintf(fp, "END: %s\n", ptest_dir);
// fprintf(fp, "%s\n", get_stime(stime, GET_STIME_BUF_SIZE));
/* Let non-master gracefully terminate */
exit(0);
}
PTEST_LIST_ITERATE_END;
- /* Reap all children before continuing */
- while (1) {
- int status;
- pid = waitpid((pid_t)(-1), &status, WNOHANG);
-
- /* Child reaped */
- if (pid > 0) {
- if (WIFEXITED(status)) {
- if (WEXITSTATUS(status) == 0)
- continue;
- }
- fprintf(fp, "One child failed to do its job pid:%u", pid);
- exit(-1);
- }
-
- /* Still children alive */
- if (pid == 0) {
- usleep(100000); /* 100ms */
- continue;
- }
-
- /* No more children to reap */
- if (pid < 0)
- break;
- }
-
+ reap_all_children(fp);
+
fprintf(fp, "START: %s\n", progname);
PTEST_LIST_ITERATE_START(head, p);
char *ptest_dir;
@@ -487,19 +502,22 @@ run_ptests(struct ptest_list *head, const struct ptest_options opts,
int status;
int fds[2]; fds[0] = pipefd_stdout[0]; fds[1] = pipefd_stderr[0];
FILE *fps[2]; fps[0] = fp; fps[1] = fp_stderr;
+ char buf[WAIT_CHILD_BUF_MAX_SIZE];
+ memset(buf, 0, sizeof(buf));
fprintf(fp, "%s\n", get_stime(stime, GET_STIME_BUF_SIZE));
fprintf(fp, "BEGIN: %s\n", ptest_dir);
status = wait_child(ptest_dir, p->run_ptest, child,
- opts.timeout, fds, fps);
+ opts.timeout, fds, fps, buf);
if (status)
rc += 1;
if (opts.xml_filename)
xml_add_case(xh,
status,
- (struct ptest_entry *)p);
+ (struct ptest_entry *)p,
+ buf);
fprintf(fp, "END: %s\n", ptest_dir);
fprintf(fp, "%s\n", get_stime(stime, GET_STIME_BUF_SIZE));
@@ -538,7 +556,7 @@ xml_create(int test_count, char *xml_filename)
}
void
-xml_add_case(FILE *xh, int status, struct ptest_entry *ptest)
+xml_add_case(FILE *xh, int status, struct ptest_entry *ptest, char *buf)
{
struct flock lock;
int fd;
@@ -556,10 +574,13 @@ xml_add_case(FILE *xh, int status, struct ptest_entry *ptest)
"\t\t<failure type='exit_code'" \
" message='run-ptest exited with code: %d'>" \
"</failure>\n" \
+ "\t\t<system-out>%s" \
+ "</system-out>\n" \
"\t</testcase>\n",
ptest->ptest,
basename(bname),
- status);
+ status,
+ buf);
}
else {
fprintf(xh, "\t<testcase classname='%s' name='%s'>\n" \
diff --git a/utils.h b/utils.h
index d0ef735..70e2abe 100644
--- a/utils.h
+++ b/utils.h
@@ -34,6 +34,7 @@ struct ptest_options {
char *directory;
int list;
int timeout;
+ int max_parallel;
char **ptests;
char *xml_filename;
};
@@ -47,7 +48,7 @@ extern int run_ptests(struct ptest_list *, const struct ptest_options,
const char *, FILE *, FILE *);
extern FILE *xml_create(int, char *);
-extern void xml_add_case(FILE *, int, struct ptest_entry *);
+extern void xml_add_case(FILE *, int, struct ptest_entry *, char *);
extern void xml_finish(FILE *);
#endif
--
1.8.3.1
^ permalink raw reply related [flat|nested] 12+ messages in thread* Re: [ptest-runner 6/8] Add <system-out></system-out> to XML when tests fail
2017-09-29 2:09 ` [ptest-runner 6/8] Add <system-out></system-out> to XML when tests fail Jiwei Sun
@ 2017-10-03 15:31 ` Joshua Lock
0 siblings, 0 replies; 12+ messages in thread
From: Joshua Lock @ 2017-10-03 15:31 UTC (permalink / raw)
To: Jiwei Sun, yocto
On 29/09/17 03:09, Jiwei Sun wrote:
> * Introduce a maximum number of concurrent ptests, since
> we are dealing with embedded targets.
This piece seems like it should be squashed into "[ptest-runner 5/8] Add
parallelism to TC execution"
There's a lot of code moved around and whitespace changes related to 5/8
in this patch too.
>
> * Add stdout || stderr output from ptests into XML.
> For testcases running in parallel, cluttering stdout won't
> help anyone.
This sounds like it could make the XML files large and unwieldy? Could
you provide some before/after numbers for the ptests in OE-Core?
Thanks,
Joshua
>
> Signed-off-by: Jiwei Sun <jiwei.sun@windriver.com>
> ---
> main.c | 2 +
> tests/data/reference.xml | 1 +
> tests/utils.c | 4 +-
> utils.c | 99 +++++++++++++++++++++++++++++-------------------
> utils.h | 3 +-
> 5 files changed, 67 insertions(+), 42 deletions(-)
>
> diff --git a/main.c b/main.c
> index d524200..a201e87 100644
> --- a/main.c
> +++ b/main.c
> @@ -37,6 +37,7 @@
>
> #define DEFAULT_DIRECTORY "/usr/lib"
> #define DEFAULT_TIMEOUT 300
> +#define DEFAULT_MAX_PARALLEL 64
>
> static inline void
> print_usage(FILE *stream, char *progname)
> @@ -69,6 +70,7 @@ main(int argc, char *argv[])
> opts.directory = strdup(DEFAULT_DIRECTORY);
> opts.list = 0;
> opts.timeout = DEFAULT_TIMEOUT;
> + opts.max_parallel = DEFAULT_MAX_PARALLEL;
> opts.ptests = NULL;
> opts.xml_filename = NULL;
>
> diff --git a/tests/data/reference.xml b/tests/data/reference.xml
> index 91522c7..a23b44b 100644
> --- a/tests/data/reference.xml
> +++ b/tests/data/reference.xml
> @@ -4,5 +4,6 @@
> </testcase>
> <testcase classname='test2' name='run-ptest'>
> <failure type='exit_code' message='run-ptest exited with code: 1'></failure>
> + <system-out>ERROR</system-out>
> </testcase>
> </testsuite>
> diff --git a/tests/utils.c b/tests/utils.c
> index 9b1f442..bb799f4 100644
> --- a/tests/utils.c
> +++ b/tests/utils.c
> @@ -262,9 +262,9 @@ START_TEST(test_xml_pass)
> ck_assert(xp != NULL);
> entry.ptest = "test1";
> entry.run_ptest = "run-ptest";
> - xml_add_case(xp, 0, &entry);
> + xml_add_case(xp, 0, &entry, "");
> entry.ptest = "test2";
> - xml_add_case(xp, 1, &entry);
> + xml_add_case(xp, 1, &entry, "ERROR");
> xml_finish(xp);
>
> FILE *fp, *fr;
> diff --git a/utils.c b/utils.c
> index 056d7b7..0e97ac4 100644
> --- a/utils.c
> +++ b/utils.c
> @@ -289,8 +289,13 @@ run_child(char *run_ptest, int fd_stdout, int fd_stderr)
> }
>
> static inline int
> -wait_child(const char *ptest_dir, const char *run_ptest, pid_t pid,
> - int timeout, int *fds, FILE **fps)
> +wait_child(const char *ptest_dir,
> + const char *run_ptest,
> + pid_t pid,
> + int timeout,
> + int *fds,
> + FILE **fps,
> + char *buf)
> {
> struct pollfd pfds[2];
> time_t sentinel;
> @@ -311,7 +316,6 @@ wait_child(const char *ptest_dir, const char *run_ptest, pid_t pid,
>
> r = poll(pfds, 2, WAIT_CHILD_POLL_TIMEOUT_MS);
> if (r > 0) {
> - char buf[WAIT_CHILD_BUF_MAX_SIZE];
> ssize_t n;
>
> if (pfds[0].revents != 0) {
> @@ -354,6 +358,35 @@ wait_child(const char *ptest_dir, const char *run_ptest, pid_t pid,
> return status;
> }
>
> +static void reap_all_children(FILE *fp) {
> + /* Reap all children before continuing */
> + while (1) {
> + int status;
> + pid_t pid;
> + pid = waitpid((pid_t)(-1), &status, WNOHANG);
> +
> + /* Child reaped */
> + if (pid > 0) {
> + if (WIFEXITED(status)) {
> + if (WEXITSTATUS(status) == 0)
> + continue;
> + }
> + fprintf(fp, "One child failed to do its job pid:%u", pid);
> + exit(-1);
> + }
> +
> + /* Still children alive */
> + if (pid == 0) {
> + usleep(100000); /* 100ms */
> + continue;
> + }
> +
> + /* No more children to reap */
> + if (pid < 0)
> + break;
> + }
> +}
> +
> int
> run_ptests(struct ptest_list *head, const struct ptest_options opts,
> const char *progname, FILE *fp, FILE *fp_stderr)
> @@ -363,8 +396,8 @@ run_ptests(struct ptest_list *head, const struct ptest_options opts,
>
> struct ptest_list *p;
> char stime[GET_STIME_BUF_SIZE];
> -
> - pid_t child, pid;
> + int nr_children = 0;
> + pid_t child;
> int pipefd_stdout[2];
> int pipefd_stderr[2];
>
> @@ -395,8 +428,13 @@ run_ptests(struct ptest_list *head, const struct ptest_options opts,
> continue;
>
> master = fork();
> - if (master)
> + if (master) {
> + if (++nr_children >= opts.max_parallel) {
> + reap_all_children(fp);
> + nr_children = 0;
> + }
> continue;
> + }
>
> ptest_dir = strdup(p->run_ptest);
> if (ptest_dir == NULL) {
> @@ -416,53 +454,30 @@ run_ptests(struct ptest_list *head, const struct ptest_options opts,
> int status;
> int fds[2]; fds[0] = pipefd_stdout[0]; fds[1] = pipefd_stderr[0];
> FILE *fps[2]; fps[0] = fp; fps[1] = fp_stderr;
> + char buf[WAIT_CHILD_BUF_MAX_SIZE];
>
> // fprintf(fp, "%s\n", get_stime(stime, GET_STIME_BUF_SIZE));
> fprintf(fp, "BEGIN: %s\n", ptest_dir);
>
> status = wait_child(ptest_dir, p->run_ptest, child,
> - opts.timeout, fds, fps);
> + opts.timeout, fds, fps, buf);
> if (status)
> rc += 1;
>
> if (opts.xml_filename)
> xml_add_case(xh,
> status,
> - (struct ptest_entry *)p);
> + (struct ptest_entry *)p,
> + buf);
>
> -// fprintf(fp, "END: %s\n", ptest_dir);
> // fprintf(fp, "%s\n", get_stime(stime, GET_STIME_BUF_SIZE));
> /* Let non-master gracefully terminate */
> exit(0);
> }
> PTEST_LIST_ITERATE_END;
>
> - /* Reap all children before continuing */
> - while (1) {
> - int status;
> - pid = waitpid((pid_t)(-1), &status, WNOHANG);
> -
> - /* Child reaped */
> - if (pid > 0) {
> - if (WIFEXITED(status)) {
> - if (WEXITSTATUS(status) == 0)
> - continue;
> - }
> - fprintf(fp, "One child failed to do its job pid:%u", pid);
> - exit(-1);
> - }
> -
> - /* Still children alive */
> - if (pid == 0) {
> - usleep(100000); /* 100ms */
> - continue;
> - }
> -
> - /* No more children to reap */
> - if (pid < 0)
> - break;
> - }
> -
> + reap_all_children(fp);
> +
> fprintf(fp, "START: %s\n", progname);
> PTEST_LIST_ITERATE_START(head, p);
> char *ptest_dir;
> @@ -487,19 +502,22 @@ run_ptests(struct ptest_list *head, const struct ptest_options opts,
> int status;
> int fds[2]; fds[0] = pipefd_stdout[0]; fds[1] = pipefd_stderr[0];
> FILE *fps[2]; fps[0] = fp; fps[1] = fp_stderr;
> + char buf[WAIT_CHILD_BUF_MAX_SIZE];
>
> + memset(buf, 0, sizeof(buf));
> fprintf(fp, "%s\n", get_stime(stime, GET_STIME_BUF_SIZE));
> fprintf(fp, "BEGIN: %s\n", ptest_dir);
>
> status = wait_child(ptest_dir, p->run_ptest, child,
> - opts.timeout, fds, fps);
> + opts.timeout, fds, fps, buf);
> if (status)
> rc += 1;
>
> if (opts.xml_filename)
> xml_add_case(xh,
> status,
> - (struct ptest_entry *)p);
> + (struct ptest_entry *)p,
> + buf);
>
> fprintf(fp, "END: %s\n", ptest_dir);
> fprintf(fp, "%s\n", get_stime(stime, GET_STIME_BUF_SIZE));
> @@ -538,7 +556,7 @@ xml_create(int test_count, char *xml_filename)
> }
>
> void
> -xml_add_case(FILE *xh, int status, struct ptest_entry *ptest)
> +xml_add_case(FILE *xh, int status, struct ptest_entry *ptest, char *buf)
> {
> struct flock lock;
> int fd;
> @@ -556,10 +574,13 @@ xml_add_case(FILE *xh, int status, struct ptest_entry *ptest)
> "\t\t<failure type='exit_code'" \
> " message='run-ptest exited with code: %d'>" \
> "</failure>\n" \
> + "\t\t<system-out>%s" \
> + "</system-out>\n" \
> "\t</testcase>\n",
> ptest->ptest,
> basename(bname),
> - status);
> + status,
> + buf);
> }
> else {
> fprintf(xh, "\t<testcase classname='%s' name='%s'>\n" \
> diff --git a/utils.h b/utils.h
> index d0ef735..70e2abe 100644
> --- a/utils.h
> +++ b/utils.h
> @@ -34,6 +34,7 @@ struct ptest_options {
> char *directory;
> int list;
> int timeout;
> + int max_parallel;
> char **ptests;
> char *xml_filename;
> };
> @@ -47,7 +48,7 @@ extern int run_ptests(struct ptest_list *, const struct ptest_options,
> const char *, FILE *, FILE *);
>
> extern FILE *xml_create(int, char *);
> -extern void xml_add_case(FILE *, int, struct ptest_entry *);
> +extern void xml_add_case(FILE *, int, struct ptest_entry *, char *);
> extern void xml_finish(FILE *);
>
> #endif
>
^ permalink raw reply [flat|nested] 12+ messages in thread
* [ptest-runner 7/8] Introduce a ret-code for timeouts
2017-09-29 2:09 [ptest-runner 1/8] Corrected return code interpreter Jiwei Sun
` (4 preceding siblings ...)
2017-09-29 2:09 ` [ptest-runner 6/8] Add <system-out></system-out> to XML when tests fail Jiwei Sun
@ 2017-09-29 2:09 ` Jiwei Sun
2017-09-29 2:09 ` [ptest-runner 8/8] Added <system-err></system-err> to XML Jiwei Sun
2017-10-03 15:31 ` [ptest-runner 1/8] Corrected return code interpreter Joshua Lock
7 siblings, 0 replies; 12+ messages in thread
From: Jiwei Sun @ 2017-09-29 2:09 UTC (permalink / raw)
To: yocto; +Cc: anibal.limon
Signed-off-by: Jiwei Sun <jiwei.sun@windriver.com>
---
utils.c | 39 +++++++++++++++++++++++++++------------
1 file changed, 27 insertions(+), 12 deletions(-)
diff --git a/utils.c b/utils.c
index 0e97ac4..54ef89d 100644
--- a/utils.c
+++ b/utils.c
@@ -349,7 +349,7 @@ wait_child(const char *ptest_dir,
}
else if(WIFSIGNALED(status) && pid > 0) {
fprintf(fps[0], " Killed by signal\n");
- return 127;
+ return 0x800000;
}
else
fprintf(fps[0], "\nERROR: Exit status is %d\n", status);
@@ -570,17 +570,32 @@ xml_add_case(FILE *xh, int status, struct ptest_entry *ptest, char *buf)
/* fprintf should guarantee atomicity for fprintfs within the same process */
if (status != 0) {
- fprintf(xh, "\t<testcase classname='%s' name='%s'>\n" \
- "\t\t<failure type='exit_code'" \
- " message='run-ptest exited with code: %d'>" \
- "</failure>\n" \
- "\t\t<system-out>%s" \
- "</system-out>\n" \
- "\t</testcase>\n",
- ptest->ptest,
- basename(bname),
- status,
- buf);
+ if (status == 0x800000) {
+ fprintf(xh, "\t<testcase classname='%s' name='%s'>\n" \
+ "\t\t<failure type='timeout'" \
+ " message='Test HUNG, Killed due to timeout 0x%x'>" \
+ "</failure>\n" \
+ "\t\t<system-out>%s" \
+ "</system-out>\n" \
+ "\t</testcase>\n",
+ ptest->ptest,
+ basename(bname),
+ status,
+ buf);
+ }
+ else {
+ fprintf(xh, "\t<testcase classname='%s' name='%s'>\n" \
+ "\t\t<failure type='exit_code'" \
+ " message='run-ptest exited with code: %d'>" \
+ "</failure>\n" \
+ "\t\t<system-out>%s" \
+ "</system-out>\n" \
+ "\t</testcase>\n",
+ ptest->ptest,
+ basename(bname),
+ status,
+ buf);
+ }
}
else {
fprintf(xh, "\t<testcase classname='%s' name='%s'>\n" \
--
1.8.3.1
^ permalink raw reply related [flat|nested] 12+ messages in thread* [ptest-runner 8/8] Added <system-err></system-err> to XML
2017-09-29 2:09 [ptest-runner 1/8] Corrected return code interpreter Jiwei Sun
` (5 preceding siblings ...)
2017-09-29 2:09 ` [ptest-runner 7/8] Introduce a ret-code for timeouts Jiwei Sun
@ 2017-09-29 2:09 ` Jiwei Sun
2017-10-03 15:31 ` Joshua Lock
2017-10-03 15:31 ` [ptest-runner 1/8] Corrected return code interpreter Joshua Lock
7 siblings, 1 reply; 12+ messages in thread
From: Jiwei Sun @ 2017-09-29 2:09 UTC (permalink / raw)
To: yocto; +Cc: anibal.limon
Signed-off-by: Jiwei Sun <jiwei.sun@windriver.com>
---
tests/data/reference.xml | 3 ++-
tests/utils.c | 4 ++--
utils.c | 49 +++++++++++++++++++++++++++++++++---------------
utils.h | 2 +-
4 files changed, 39 insertions(+), 19 deletions(-)
diff --git a/tests/data/reference.xml b/tests/data/reference.xml
index a23b44b..0dea38c 100644
--- a/tests/data/reference.xml
+++ b/tests/data/reference.xml
@@ -4,6 +4,7 @@
</testcase>
<testcase classname='test2' name='run-ptest'>
<failure type='exit_code' message='run-ptest exited with code: 1'></failure>
- <system-out>ERROR</system-out>
+ <system-out>STDOUT ERROR</system-out>
+ <system-err>STDERR</system-err>
</testcase>
</testsuite>
diff --git a/tests/utils.c b/tests/utils.c
index bb799f4..230116a 100644
--- a/tests/utils.c
+++ b/tests/utils.c
@@ -262,9 +262,9 @@ START_TEST(test_xml_pass)
ck_assert(xp != NULL);
entry.ptest = "test1";
entry.run_ptest = "run-ptest";
- xml_add_case(xp, 0, &entry, "");
+ xml_add_case(xp, 0, &entry, "", "");
entry.ptest = "test2";
- xml_add_case(xp, 1, &entry, "ERROR");
+ xml_add_case(xp, 1, &entry, "STDOUT ERROR", "STDERR");
xml_finish(xp);
FILE *fp, *fr;
diff --git a/utils.c b/utils.c
index 54ef89d..0f712f9 100644
--- a/utils.c
+++ b/utils.c
@@ -295,7 +295,8 @@ wait_child(const char *ptest_dir,
int timeout,
int *fds,
FILE **fps,
- char *buf)
+ char *bufout,
+ char *buferr)
{
struct pollfd pfds[2];
time_t sentinel;
@@ -319,13 +320,13 @@ wait_child(const char *ptest_dir,
ssize_t n;
if (pfds[0].revents != 0) {
- while ((n = read(fds[0], buf, WAIT_CHILD_BUF_MAX_SIZE)) > 0)
- fwrite(buf, n, 1, fps[0]);
+ while ((n = read(fds[0], bufout, WAIT_CHILD_BUF_MAX_SIZE)) > 0)
+ fwrite(bufout, n, 1, fps[0]);
}
if (pfds[1].revents != 0) {
- while ((n = read(fds[1], buf, WAIT_CHILD_BUF_MAX_SIZE)) > 0)
- fwrite(buf, n, 1, fps[1]);
+ while ((n = read(fds[1], buferr, WAIT_CHILD_BUF_MAX_SIZE)) > 0)
+ fwrite(buferr, n, 1, fps[1]);
}
sentinel = time(NULL);
@@ -454,13 +455,15 @@ run_ptests(struct ptest_list *head, const struct ptest_options opts,
int status;
int fds[2]; fds[0] = pipefd_stdout[0]; fds[1] = pipefd_stderr[0];
FILE *fps[2]; fps[0] = fp; fps[1] = fp_stderr;
- char buf[WAIT_CHILD_BUF_MAX_SIZE];
+ char bufout[WAIT_CHILD_BUF_MAX_SIZE];
+ char buferr[WAIT_CHILD_BUF_MAX_SIZE];
// fprintf(fp, "%s\n", get_stime(stime, GET_STIME_BUF_SIZE));
fprintf(fp, "BEGIN: %s\n", ptest_dir);
status = wait_child(ptest_dir, p->run_ptest, child,
- opts.timeout, fds, fps, buf);
+ opts.timeout, fds, fps, bufout,
+ buferr);
if (status)
rc += 1;
@@ -468,7 +471,8 @@ run_ptests(struct ptest_list *head, const struct ptest_options opts,
xml_add_case(xh,
status,
(struct ptest_entry *)p,
- buf);
+ bufout,
+ buferr);
// fprintf(fp, "%s\n", get_stime(stime, GET_STIME_BUF_SIZE));
/* Let non-master gracefully terminate */
@@ -502,14 +506,18 @@ run_ptests(struct ptest_list *head, const struct ptest_options opts,
int status;
int fds[2]; fds[0] = pipefd_stdout[0]; fds[1] = pipefd_stderr[0];
FILE *fps[2]; fps[0] = fp; fps[1] = fp_stderr;
- char buf[WAIT_CHILD_BUF_MAX_SIZE];
+ char bufout[WAIT_CHILD_BUF_MAX_SIZE];
+ char buferr[WAIT_CHILD_BUF_MAX_SIZE];
+
+ memset(bufout, 0, sizeof(bufout));
+ memset(buferr, 0, sizeof(buferr));
- memset(buf, 0, sizeof(buf));
fprintf(fp, "%s\n", get_stime(stime, GET_STIME_BUF_SIZE));
fprintf(fp, "BEGIN: %s\n", ptest_dir);
status = wait_child(ptest_dir, p->run_ptest, child,
- opts.timeout, fds, fps, buf);
+ opts.timeout, fds, fps, bufout,
+ buferr);
if (status)
rc += 1;
@@ -517,7 +525,8 @@ run_ptests(struct ptest_list *head, const struct ptest_options opts,
xml_add_case(xh,
status,
(struct ptest_entry *)p,
- buf);
+ bufout,
+ buferr);
fprintf(fp, "END: %s\n", ptest_dir);
fprintf(fp, "%s\n", get_stime(stime, GET_STIME_BUF_SIZE));
@@ -556,7 +565,11 @@ xml_create(int test_count, char *xml_filename)
}
void
-xml_add_case(FILE *xh, int status, struct ptest_entry *ptest, char *buf)
+xml_add_case(FILE *xh,
+ int status,
+ struct ptest_entry *ptest,
+ char *bufout,
+ char *buferr)
{
struct flock lock;
int fd;
@@ -577,11 +590,14 @@ xml_add_case(FILE *xh, int status, struct ptest_entry *ptest, char *buf)
"</failure>\n" \
"\t\t<system-out>%s" \
"</system-out>\n" \
+ "\t\t<system-err>%s" \
+ "</system-err>\n" \
"\t</testcase>\n",
ptest->ptest,
basename(bname),
status,
- buf);
+ bufout,
+ buferr);
}
else {
fprintf(xh, "\t<testcase classname='%s' name='%s'>\n" \
@@ -590,11 +606,14 @@ xml_add_case(FILE *xh, int status, struct ptest_entry *ptest, char *buf)
"</failure>\n" \
"\t\t<system-out>%s" \
"</system-out>\n" \
+ "\t\t<system-err>%s" \
+ "</system-err>\n" \
"\t</testcase>\n",
ptest->ptest,
basename(bname),
status,
- buf);
+ bufout,
+ buferr);
}
}
else {
diff --git a/utils.h b/utils.h
index 70e2abe..8f2e5b4 100644
--- a/utils.h
+++ b/utils.h
@@ -48,7 +48,7 @@ extern int run_ptests(struct ptest_list *, const struct ptest_options,
const char *, FILE *, FILE *);
extern FILE *xml_create(int, char *);
-extern void xml_add_case(FILE *, int, struct ptest_entry *, char *);
+extern void xml_add_case(FILE *, int, struct ptest_entry *, char *, char *);
extern void xml_finish(FILE *);
#endif
--
1.8.3.1
^ permalink raw reply related [flat|nested] 12+ messages in thread* Re: [ptest-runner 8/8] Added <system-err></system-err> to XML
2017-09-29 2:09 ` [ptest-runner 8/8] Added <system-err></system-err> to XML Jiwei Sun
@ 2017-10-03 15:31 ` Joshua Lock
0 siblings, 0 replies; 12+ messages in thread
From: Joshua Lock @ 2017-10-03 15:31 UTC (permalink / raw)
To: Jiwei Sun, yocto; +Cc: anibal.limon
Is this an evolution of the change in 6/8? Can it be squashed in with
that patch?
Thanks,
Joshua
On 29/09/17 03:09, Jiwei Sun wrote:
> Signed-off-by: Jiwei Sun <jiwei.sun@windriver.com>
> ---
> tests/data/reference.xml | 3 ++-
> tests/utils.c | 4 ++--
> utils.c | 49 +++++++++++++++++++++++++++++++++---------------
> utils.h | 2 +-
> 4 files changed, 39 insertions(+), 19 deletions(-)
>
> diff --git a/tests/data/reference.xml b/tests/data/reference.xml
> index a23b44b..0dea38c 100644
> --- a/tests/data/reference.xml
> +++ b/tests/data/reference.xml
> @@ -4,6 +4,7 @@
> </testcase>
> <testcase classname='test2' name='run-ptest'>
> <failure type='exit_code' message='run-ptest exited with code: 1'></failure>
> - <system-out>ERROR</system-out>
> + <system-out>STDOUT ERROR</system-out>
> + <system-err>STDERR</system-err>
> </testcase>
> </testsuite>
> diff --git a/tests/utils.c b/tests/utils.c
> index bb799f4..230116a 100644
> --- a/tests/utils.c
> +++ b/tests/utils.c
> @@ -262,9 +262,9 @@ START_TEST(test_xml_pass)
> ck_assert(xp != NULL);
> entry.ptest = "test1";
> entry.run_ptest = "run-ptest";
> - xml_add_case(xp, 0, &entry, "");
> + xml_add_case(xp, 0, &entry, "", "");
> entry.ptest = "test2";
> - xml_add_case(xp, 1, &entry, "ERROR");
> + xml_add_case(xp, 1, &entry, "STDOUT ERROR", "STDERR");
> xml_finish(xp);
>
> FILE *fp, *fr;
> diff --git a/utils.c b/utils.c
> index 54ef89d..0f712f9 100644
> --- a/utils.c
> +++ b/utils.c
> @@ -295,7 +295,8 @@ wait_child(const char *ptest_dir,
> int timeout,
> int *fds,
> FILE **fps,
> - char *buf)
> + char *bufout,
> + char *buferr)
> {
> struct pollfd pfds[2];
> time_t sentinel;
> @@ -319,13 +320,13 @@ wait_child(const char *ptest_dir,
> ssize_t n;
>
> if (pfds[0].revents != 0) {
> - while ((n = read(fds[0], buf, WAIT_CHILD_BUF_MAX_SIZE)) > 0)
> - fwrite(buf, n, 1, fps[0]);
> + while ((n = read(fds[0], bufout, WAIT_CHILD_BUF_MAX_SIZE)) > 0)
> + fwrite(bufout, n, 1, fps[0]);
> }
>
> if (pfds[1].revents != 0) {
> - while ((n = read(fds[1], buf, WAIT_CHILD_BUF_MAX_SIZE)) > 0)
> - fwrite(buf, n, 1, fps[1]);
> + while ((n = read(fds[1], buferr, WAIT_CHILD_BUF_MAX_SIZE)) > 0)
> + fwrite(buferr, n, 1, fps[1]);
> }
>
> sentinel = time(NULL);
> @@ -454,13 +455,15 @@ run_ptests(struct ptest_list *head, const struct ptest_options opts,
> int status;
> int fds[2]; fds[0] = pipefd_stdout[0]; fds[1] = pipefd_stderr[0];
> FILE *fps[2]; fps[0] = fp; fps[1] = fp_stderr;
> - char buf[WAIT_CHILD_BUF_MAX_SIZE];
> + char bufout[WAIT_CHILD_BUF_MAX_SIZE];
> + char buferr[WAIT_CHILD_BUF_MAX_SIZE];
>
> // fprintf(fp, "%s\n", get_stime(stime, GET_STIME_BUF_SIZE));
> fprintf(fp, "BEGIN: %s\n", ptest_dir);
>
> status = wait_child(ptest_dir, p->run_ptest, child,
> - opts.timeout, fds, fps, buf);
> + opts.timeout, fds, fps, bufout,
> + buferr);
> if (status)
> rc += 1;
>
> @@ -468,7 +471,8 @@ run_ptests(struct ptest_list *head, const struct ptest_options opts,
> xml_add_case(xh,
> status,
> (struct ptest_entry *)p,
> - buf);
> + bufout,
> + buferr);
>
> // fprintf(fp, "%s\n", get_stime(stime, GET_STIME_BUF_SIZE));
> /* Let non-master gracefully terminate */
> @@ -502,14 +506,18 @@ run_ptests(struct ptest_list *head, const struct ptest_options opts,
> int status;
> int fds[2]; fds[0] = pipefd_stdout[0]; fds[1] = pipefd_stderr[0];
> FILE *fps[2]; fps[0] = fp; fps[1] = fp_stderr;
> - char buf[WAIT_CHILD_BUF_MAX_SIZE];
> + char bufout[WAIT_CHILD_BUF_MAX_SIZE];
> + char buferr[WAIT_CHILD_BUF_MAX_SIZE];
> +
> + memset(bufout, 0, sizeof(bufout));
> + memset(buferr, 0, sizeof(buferr));
>
> - memset(buf, 0, sizeof(buf));
> fprintf(fp, "%s\n", get_stime(stime, GET_STIME_BUF_SIZE));
> fprintf(fp, "BEGIN: %s\n", ptest_dir);
>
> status = wait_child(ptest_dir, p->run_ptest, child,
> - opts.timeout, fds, fps, buf);
> + opts.timeout, fds, fps, bufout,
> + buferr);
> if (status)
> rc += 1;
>
> @@ -517,7 +525,8 @@ run_ptests(struct ptest_list *head, const struct ptest_options opts,
> xml_add_case(xh,
> status,
> (struct ptest_entry *)p,
> - buf);
> + bufout,
> + buferr);
>
> fprintf(fp, "END: %s\n", ptest_dir);
> fprintf(fp, "%s\n", get_stime(stime, GET_STIME_BUF_SIZE));
> @@ -556,7 +565,11 @@ xml_create(int test_count, char *xml_filename)
> }
>
> void
> -xml_add_case(FILE *xh, int status, struct ptest_entry *ptest, char *buf)
> +xml_add_case(FILE *xh,
> + int status,
> + struct ptest_entry *ptest,
> + char *bufout,
> + char *buferr)
> {
> struct flock lock;
> int fd;
> @@ -577,11 +590,14 @@ xml_add_case(FILE *xh, int status, struct ptest_entry *ptest, char *buf)
> "</failure>\n" \
> "\t\t<system-out>%s" \
> "</system-out>\n" \
> + "\t\t<system-err>%s" \
> + "</system-err>\n" \
> "\t</testcase>\n",
> ptest->ptest,
> basename(bname),
> status,
> - buf);
> + bufout,
> + buferr);
> }
> else {
> fprintf(xh, "\t<testcase classname='%s' name='%s'>\n" \
> @@ -590,11 +606,14 @@ xml_add_case(FILE *xh, int status, struct ptest_entry *ptest, char *buf)
> "</failure>\n" \
> "\t\t<system-out>%s" \
> "</system-out>\n" \
> + "\t\t<system-err>%s" \
> + "</system-err>\n" \
> "\t</testcase>\n",
> ptest->ptest,
> basename(bname),
> status,
> - buf);
> + bufout,
> + buferr);
> }
> }
> else {
> diff --git a/utils.h b/utils.h
> index 70e2abe..8f2e5b4 100644
> --- a/utils.h
> +++ b/utils.h
> @@ -48,7 +48,7 @@ extern int run_ptests(struct ptest_list *, const struct ptest_options,
> const char *, FILE *, FILE *);
>
> extern FILE *xml_create(int, char *);
> -extern void xml_add_case(FILE *, int, struct ptest_entry *, char *);
> +extern void xml_add_case(FILE *, int, struct ptest_entry *, char *, char *);
> extern void xml_finish(FILE *);
>
> #endif
>
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [ptest-runner 1/8] Corrected return code interpreter
2017-09-29 2:09 [ptest-runner 1/8] Corrected return code interpreter Jiwei Sun
` (6 preceding siblings ...)
2017-09-29 2:09 ` [ptest-runner 8/8] Added <system-err></system-err> to XML Jiwei Sun
@ 2017-10-03 15:31 ` Joshua Lock
7 siblings, 0 replies; 12+ messages in thread
From: Joshua Lock @ 2017-10-03 15:31 UTC (permalink / raw)
To: Jiwei Sun, yocto
Can you add a commit message detailing what changed and why? Most of the
commit messages in this series don't help me understand what changed and
why: https://chris.beams.io/posts/git-commit/
This is especially important as the original author of the code has
moved on from the project.
On 29/09/17 03:09, Jiwei Sun wrote:
> Signed-off-by: Jiwei Sun <jiwei.sun@windriver.com>
> ---
> utils.c | 14 ++++++++++++--
> 1 file changed, 12 insertions(+), 2 deletions(-)
>
> diff --git a/utils.c b/utils.c
> index 6d65388..a07faec 100644
> --- a/utils.c
> +++ b/utils.c
> @@ -300,9 +300,19 @@ wait_child(const char *ptest_dir, const char *run_ptest, pid_t pid,
> }
>
> if (status) {
> - fprintf(fps[0], "\nERROR: Exit status is %d\n", status);
> if (timeouted)
> - fprintf(fps[0], "TIMEOUT: %s\n", ptest_dir);
> + fprintf(fps[0], "TIMEOUT: %s ", ptest_dir);
> +
> + if(WIFEXITED(status)) {
> + fprintf(fps[0], "\nERROR: Exit status is %d\n", WEXITSTATUS(status));
> + return WEXITSTATUS(status);
> + }
> + else if(WIFSIGNALED(status)) {
> + fprintf(fps[0], " Killed by signal\n");
> + return 127;
Why 127? That's "command not found" in bash and EKEYEXPIRED in my errno.h
Could we just include errno and use a named error code?
> + }
> + else
> + fprintf(fps[0], "\nERROR: Exit status is %d\n", status);
> }
>
> return status;
>
Thanks,
Joshua
^ permalink raw reply [flat|nested] 12+ messages in thread