[PATCH 0/4] tools/nolibc: add testcases for vfprintf

[PATCH 0/4] tools/nolibc: add testcases for vfprintf

[Willy Tarreau]

Hello Paul,

Thomas added new tests for vfprintf(), which is a good idea because it
was a new implementation, we use it a lot in the tests so we'd rather
make sure it works! This required to implement support for memfd_create()
that is used to collect the output into a buffer, as well as to complete
a little bit the minimalistic FILE emulation with fileno(), fdopen(),
fflush() and fclose(). The result is neat and works equally on glibc and
nolibc. We just had to cheat on the pointer test because for NULL nolibc
prints "0x0" while glibc prints "(nil)" so we check 0x1 instead to avoid
this special case.

Finally Thomas added a new target to the makefile to ease building the
test against the default libc. This should help detect incompatibilities
when new features are added.

I've tested it locally with my libc and against all supported architectures
(userland tests only), and all tests passed.

This can be added to your dev tree for 6.5 on top of the previous series.

Thanks!
Willy

Thomas Weißschuh (4):
  tools/nolibc: add libc-test binary
  tools/nolibc: add wrapper for memfd_create
  tools/nolibc: implement fd-based FILE streams
  tools/nolibc: add testcases for vfprintf

 tools/include/nolibc/stdio.h                 | 95 ++++++++++++++------
 tools/include/nolibc/sys.h                   | 23 +++++
 tools/testing/selftests/nolibc/.gitignore    |  1 +
 tools/testing/selftests/nolibc/Makefile      |  6 ++
 tools/testing/selftests/nolibc/nolibc-test.c | 86 ++++++++++++++++++
 5 files changed, 184 insertions(+), 27 deletions(-)

-- 
2.17.5

[PATCH 1/4] tools/nolibc: add libc-test binary

[Willy Tarreau]

From: Thomas Weißschuh <linux@weissschuh.net>

This can be used to easily compare the behavior of nolibc to the system
libc.

Signed-off-by: Thomas Weißschuh <linux@weissschuh.net>
Signed-off-by: Willy Tarreau <w@1wt.eu>
---
 tools/testing/selftests/nolibc/.gitignore | 1 +
 tools/testing/selftests/nolibc/Makefile   | 6 ++++++
 2 files changed, 7 insertions(+)

diff --git a/tools/testing/selftests/nolibc/.gitignore b/tools/testing/selftests/nolibc/.gitignore
index 4696df589d68..52f613cdad54 100644
--- a/tools/testing/selftests/nolibc/.gitignore
+++ b/tools/testing/selftests/nolibc/.gitignore
@@ -1,4 +1,5 @@
 /initramfs/
+/libc-test
 /nolibc-test
 /run.out
 /sysroot/
diff --git a/tools/testing/selftests/nolibc/Makefile b/tools/testing/selftests/nolibc/Makefile
index bbce57420465..0cbe13809b37 100644
--- a/tools/testing/selftests/nolibc/Makefile
+++ b/tools/testing/selftests/nolibc/Makefile
@@ -94,6 +94,7 @@ help:
 	@echo "  help         this help"
 	@echo "  sysroot      create the nolibc sysroot here (uses \$$ARCH)"
 	@echo "  nolibc-test  build the executable (uses \$$CC and \$$CROSS_COMPILE)"
+	@echo "  libc-test    build an executable using the compiler's default libc instead"
 	@echo "  run-user     runs the executable under QEMU (uses \$$ARCH, \$$TEST)"
 	@echo "  initramfs    prepare the initramfs with nolibc-test"
 	@echo "  defconfig    create a fresh new default config (uses \$$ARCH)"
@@ -128,6 +129,9 @@ nolibc-test: nolibc-test.c sysroot/$(ARCH)/include
 	$(QUIET_CC)$(CC) $(CFLAGS) $(LDFLAGS) -o $@ \
 	  -nostdlib -static -Isysroot/$(ARCH)/include $< -lgcc
 
+libc-test: nolibc-test.c
+	$(QUIET_CC)$(CC) -o $@ $<
+
 # qemu user-land test
 run-user: nolibc-test
 	$(Q)qemu-$(QEMU_ARCH) ./nolibc-test > "$(CURDIR)/run.out" || :
@@ -159,6 +163,8 @@ clean:
 	$(Q)rm -rf sysroot
 	$(call QUIET_CLEAN, nolibc-test)
 	$(Q)rm -f nolibc-test
+	$(call QUIET_CLEAN, libc-test)
+	$(Q)rm -f libc-test
 	$(call QUIET_CLEAN, initramfs)
 	$(Q)rm -rf initramfs
 	$(call QUIET_CLEAN, run.out)
-- 
2.17.5

[PATCH 2/4] tools/nolibc: add wrapper for memfd_create

[Willy Tarreau]

From: Thomas Weißschuh <linux@weissschuh.net>

This is useful for users and will also be used by a future testcase.

Signed-off-by: Thomas Weißschuh <linux@weissschuh.net>
Signed-off-by: Willy Tarreau <w@1wt.eu>
---
 tools/include/nolibc/sys.h | 23 +++++++++++++++++++++++
 1 file changed, 23 insertions(+)

diff --git a/tools/include/nolibc/sys.h b/tools/include/nolibc/sys.h
index 5d624dc63a42..bea9760dbd16 100644
--- a/tools/include/nolibc/sys.h
+++ b/tools/include/nolibc/sys.h
@@ -1365,6 +1365,29 @@ ssize_t write(int fd, const void *buf, size_t count)
 	return ret;
 }
 
+
+/*
+ * int memfd_create(const char *name, unsigned int flags);
+ */
+
+static __attribute__((unused))
+int sys_memfd_create(const char *name, unsigned int flags)
+{
+	return my_syscall2(__NR_memfd_create, name, flags);
+}
+
+static __attribute__((unused))
+int memfd_create(const char *name, unsigned int flags)
+{
+	ssize_t ret = sys_memfd_create(name, flags);
+
+	if (ret < 0) {
+		SET_ERRNO(-ret);
+		ret = -1;
+	}
+	return ret;
+}
+
 /* make sure to include all global symbols */
 #include "nolibc.h"
 
-- 
2.17.5

[PATCH 3/4] tools/nolibc: implement fd-based FILE streams

[Willy Tarreau]

From: Thomas Weißschuh <linux@weissschuh.net>

This enables the usage of the stream APIs with arbitrary filedescriptors.

It will be used by a future testcase.

Signed-off-by: Thomas Weißschuh <linux@weissschuh.net>
Signed-off-by: Willy Tarreau <w@1wt.eu>
---
 tools/include/nolibc/stdio.h | 95 ++++++++++++++++++++++++++----------
 1 file changed, 68 insertions(+), 27 deletions(-)

diff --git a/tools/include/nolibc/stdio.h b/tools/include/nolibc/stdio.h
index 96ac8afc5aee..4add736c07aa 100644
--- a/tools/include/nolibc/stdio.h
+++ b/tools/include/nolibc/stdio.h
@@ -21,17 +21,75 @@
 #define EOF (-1)
 #endif
 
-/* just define FILE as a non-empty type */
+/* just define FILE as a non-empty type. The value of the pointer gives
+ * the FD: FILE=~fd for fd>=0 or NULL for fd<0. This way positive FILE
+ * are immediately identified as abnormal entries (i.e. possible copies
+ * of valid pointers to something else).
+ */
 typedef struct FILE {
 	char dummy[1];
 } FILE;
 
-/* We define the 3 common stdio files as constant invalid pointers that
- * are easily recognized.
- */
-static __attribute__((unused)) FILE* const stdin  = (FILE*)-3;
-static __attribute__((unused)) FILE* const stdout = (FILE*)-2;
-static __attribute__((unused)) FILE* const stderr = (FILE*)-1;
+static __attribute__((unused)) FILE* const stdin  = (FILE*)(intptr_t)~STDIN_FILENO;
+static __attribute__((unused)) FILE* const stdout = (FILE*)(intptr_t)~STDOUT_FILENO;
+static __attribute__((unused)) FILE* const stderr = (FILE*)(intptr_t)~STDERR_FILENO;
+
+/* provides a FILE* equivalent of fd. The mode is ignored. */
+static __attribute__((unused))
+FILE *fdopen(int fd, const char *mode __attribute__((unused)))
+{
+	if (fd < 0) {
+		SET_ERRNO(EBADF);
+		return NULL;
+	}
+	return (FILE*)(intptr_t)~fd;
+}
+
+/* provides the fd of stream. */
+static __attribute__((unused))
+int fileno(FILE *stream)
+{
+	intptr_t i = (intptr_t)stream;
+
+	if (i >= 0) {
+		SET_ERRNO(EBADF);
+		return -1;
+	}
+	return ~i;
+}
+
+/* flush a stream. */
+static __attribute__((unused))
+int fflush(FILE *stream)
+{
+	intptr_t i = (intptr_t)stream;
+
+	/* NULL is valid here. */
+	if (i > 0) {
+		SET_ERRNO(EBADF);
+		return -1;
+	}
+
+	/* Don't do anything, nolibc does not support buffering. */
+	return 0;
+}
+
+/* flush a stream. */
+static __attribute__((unused))
+int fclose(FILE *stream)
+{
+	intptr_t i = (intptr_t)stream;
+
+	if (i >= 0) {
+		SET_ERRNO(EBADF);
+		return -1;
+	}
+
+	if (close(~i))
+		return EOF;
+
+	return 0;
+}
 
 /* getc(), fgetc(), getchar() */
 
@@ -41,14 +99,8 @@ static __attribute__((unused))
 int fgetc(FILE* stream)
 {
 	unsigned char ch;
-	int fd;
 
-	if (stream < stdin || stream > stderr)
-		return EOF;
-
-	fd = 3 + (long)stream;
-
-	if (read(fd, &ch, 1) <= 0)
+	if (read(fileno(stream), &ch, 1) <= 0)
 		return EOF;
 	return ch;
 }
@@ -68,14 +120,8 @@ static __attribute__((unused))
 int fputc(int c, FILE* stream)
 {
 	unsigned char ch = c;
-	int fd;
-
-	if (stream < stdin || stream > stderr)
-		return EOF;
-
-	fd = 3 + (long)stream;
 
-	if (write(fd, &ch, 1) <= 0)
+	if (write(fileno(stream), &ch, 1) <= 0)
 		return EOF;
 	return ch;
 }
@@ -96,12 +142,7 @@ static __attribute__((unused))
 int _fwrite(const void *buf, size_t size, FILE *stream)
 {
 	ssize_t ret;
-	int fd;
-
-	if (stream < stdin || stream > stderr)
-		return EOF;
-
-	fd = 3 + (long)stream;
+	int fd = fileno(stream);
 
 	while (size) {
 		ret = write(fd, buf, size);
-- 
2.17.5

[PATCH 4/4] tools/nolibc: add testcases for vfprintf

[Willy Tarreau]

From: Thomas Weißschuh <linux@weissschuh.net>

vfprintf() is complex and so far did not have proper tests.

Signed-off-by: Thomas Weißschuh <linux@weissschuh.net>
Signed-off-by: Willy Tarreau <w@1wt.eu>
---
 tools/testing/selftests/nolibc/nolibc-test.c | 86 ++++++++++++++++++++
 1 file changed, 86 insertions(+)

diff --git a/tools/testing/selftests/nolibc/nolibc-test.c b/tools/testing/selftests/nolibc/nolibc-test.c
index 1bafbd8da6af..888da60eb5ba 100644
--- a/tools/testing/selftests/nolibc/nolibc-test.c
+++ b/tools/testing/selftests/nolibc/nolibc-test.c
@@ -20,6 +20,7 @@
 #include <linux/reboot.h>
 #include <sys/io.h>
 #include <sys/ioctl.h>
+#include <sys/mman.h>
 #include <sys/mount.h>
 #include <sys/reboot.h>
 #include <sys/stat.h>
@@ -669,6 +670,90 @@ int run_stdlib(int min, int max)
 	return ret;
 }
 
+#define EXPECT_VFPRINTF(c, expected, fmt, ...)				\
+	ret += expect_vfprintf(llen, c, expected, fmt, ##__VA_ARGS__)
+
+static int expect_vfprintf(int llen, size_t c, const char *expected, const char *fmt, ...)
+{
+	int ret, fd, w, r;
+	char buf[100];
+	FILE *memfile;
+	va_list args;
+
+	fd = memfd_create("vfprintf", 0);
+	if (fd == -1) {
+		pad_spc(llen, 64, "[FAIL]\n");
+		return 1;
+	}
+
+	memfile = fdopen(fd, "w+");
+	if (!memfile) {
+		pad_spc(llen, 64, "[FAIL]\n");
+		return 1;
+	}
+
+	va_start(args, fmt);
+	w = vfprintf(memfile, fmt, args);
+	va_end(args);
+
+	if (w != c) {
+		llen += printf(" written(%d) != %d", w, (int) c);
+		pad_spc(llen, 64, "[FAIL]\n");
+		return 1;
+	}
+
+	fflush(memfile);
+	lseek(fd, 0, SEEK_SET);
+
+	r = read(fd, buf, sizeof(buf) - 1);
+	buf[r] = '\0';
+
+	fclose(memfile);
+
+	if (r != w) {
+		llen += printf(" written(%d) != read(%d)", w, r);
+		pad_spc(llen, 64, "[FAIL]\n");
+		return 1;
+	}
+
+	llen += printf(" \"%s\" = \"%s\"", expected, buf);
+	ret = strncmp(expected, buf, c);
+
+	pad_spc(llen, 64, ret ? "[FAIL]\n" : " [OK]\n");
+	return ret;
+}
+
+static int run_vfprintf(int min, int max)
+{
+	int test;
+	int tmp;
+	int ret = 0;
+	void *p1, *p2;
+
+	for (test = min; test >= 0 && test <= max; test++) {
+		int llen = 0; // line length
+
+		/* avoid leaving empty lines below, this will insert holes into
+		 * test numbers.
+		 */
+		switch (test + __LINE__ + 1) {
+		CASE_TEST(empty);        EXPECT_VFPRINTF(0, "", ""); break;
+		CASE_TEST(simple);       EXPECT_VFPRINTF(3, "foo", "foo"); break;
+		CASE_TEST(string);       EXPECT_VFPRINTF(3, "foo", "%s", "foo"); break;
+		CASE_TEST(number);       EXPECT_VFPRINTF(4, "1234", "%d", 1234); break;
+		CASE_TEST(negnumber);    EXPECT_VFPRINTF(5, "-1234", "%d", -1234); break;
+		CASE_TEST(unsigned);     EXPECT_VFPRINTF(5, "12345", "%u", 12345); break;
+		CASE_TEST(char);         EXPECT_VFPRINTF(1, "c", "%c", 'c'); break;
+		CASE_TEST(hex);          EXPECT_VFPRINTF(1, "f", "%x", 0xf); break;
+		CASE_TEST(pointer);      EXPECT_VFPRINTF(3, "0x1", "%p", (void *) 0x1); break;
+		case __LINE__:
+			return ret; /* must be last */
+		/* note: do not set any defaults so as to permit holes above */
+		}
+	}
+	return ret;
+}
+
 static int smash_stack(void)
 {
 	char buf[100];
@@ -777,6 +862,7 @@ static const struct test test_names[] = {
 	/* add new tests here */
 	{ .name = "syscall",    .func = run_syscall    },
 	{ .name = "stdlib",     .func = run_stdlib     },
+	{ .name = "vfprintf",   .func = run_vfprintf   },
 	{ .name = "protection", .func = run_protection },
 	{ 0 }
 };
-- 
2.17.5

Re: [PATCH 0/4] tools/nolibc: add testcases for vfprintf

[Paul E. McKenney]

On Sun, Apr 02, 2023 at 08:48:02PM +0200, Willy Tarreau wrote:
> Hello Paul,
> 
> Thomas added new tests for vfprintf(), which is a good idea because it
> was a new implementation, we use it a lot in the tests so we'd rather
> make sure it works! This required to implement support for memfd_create()
> that is used to collect the output into a buffer, as well as to complete
> a little bit the minimalistic FILE emulation with fileno(), fdopen(),
> fflush() and fclose(). The result is neat and works equally on glibc and
> nolibc. We just had to cheat on the pointer test because for NULL nolibc
> prints "0x0" while glibc prints "(nil)" so we check 0x1 instead to avoid
> this special case.
> 
> Finally Thomas added a new target to the makefile to ease building the
> test against the default libc. This should help detect incompatibilities
> when new features are added.
> 
> I've tested it locally with my libc and against all supported architectures
> (userland tests only), and all tests passed.
> 
> This can be added to your dev tree for 6.5 on top of the previous series.

Looks like some useful code to make testing more comprehensive, thank
you both!  Queued and pushed.

							Thanx, Paul

> Thanks!
> Willy
> 
> Thomas Weißschuh (4):
>   tools/nolibc: add libc-test binary
>   tools/nolibc: add wrapper for memfd_create
>   tools/nolibc: implement fd-based FILE streams
>   tools/nolibc: add testcases for vfprintf
> 
>  tools/include/nolibc/stdio.h                 | 95 ++++++++++++++------
>  tools/include/nolibc/sys.h                   | 23 +++++
>  tools/testing/selftests/nolibc/.gitignore    |  1 +
>  tools/testing/selftests/nolibc/Makefile      |  6 ++
>  tools/testing/selftests/nolibc/nolibc-test.c | 86 ++++++++++++++++++
>  5 files changed, 184 insertions(+), 27 deletions(-)
> 
> -- 
> 2.17.5
> 

Re: [PATCH 0/4] tools/nolibc: add testcases for vfprintf

[Paul E. McKenney]

On Sun, Apr 02, 2023 at 02:24:04PM -0700, Paul E. McKenney wrote:
> On Sun, Apr 02, 2023 at 08:48:02PM +0200, Willy Tarreau wrote:
> > Hello Paul,
> > 
> > Thomas added new tests for vfprintf(), which is a good idea because it
> > was a new implementation, we use it a lot in the tests so we'd rather
> > make sure it works! This required to implement support for memfd_create()
> > that is used to collect the output into a buffer, as well as to complete
> > a little bit the minimalistic FILE emulation with fileno(), fdopen(),
> > fflush() and fclose(). The result is neat and works equally on glibc and
> > nolibc. We just had to cheat on the pointer test because for NULL nolibc
> > prints "0x0" while glibc prints "(nil)" so we check 0x1 instead to avoid
> > this special case.
> > 
> > Finally Thomas added a new target to the makefile to ease building the
> > test against the default libc. This should help detect incompatibilities
> > when new features are added.
> > 
> > I've tested it locally with my libc and against all supported architectures
> > (userland tests only), and all tests passed.
> > 
> > This can be added to your dev tree for 6.5 on top of the previous series.
> 
> Looks like some useful code to make testing more comprehensive, thank
> you both!  Queued and pushed.

And finally tested:

Kernel: arch/x86/boot/bzImage is ready  (#7)
make[1]: Leaving directory '/home/git/linux-build'
133 test(s) passed.

  CC      nolibc-test
133 test(s) passed.

So good agreement between user and kernel, anyway.  Usermode tests
note that two of the tests are skipped (chroot_root and link_blah).
Kernelmode tests all say "[OK]".

							Thanx, Paul

Re: [PATCH 0/4] tools/nolibc: add testcases for vfprintf

[Willy Tarreau]

On Tue, Apr 04, 2023 at 01:40:32PM -0700, Paul E. McKenney wrote:
> On Sun, Apr 02, 2023 at 02:24:04PM -0700, Paul E. McKenney wrote:
> > On Sun, Apr 02, 2023 at 08:48:02PM +0200, Willy Tarreau wrote:
> > > Hello Paul,
> > > 
> > > Thomas added new tests for vfprintf(), which is a good idea because it
> > > was a new implementation, we use it a lot in the tests so we'd rather
> > > make sure it works! This required to implement support for memfd_create()
> > > that is used to collect the output into a buffer, as well as to complete
> > > a little bit the minimalistic FILE emulation with fileno(), fdopen(),
> > > fflush() and fclose(). The result is neat and works equally on glibc and
> > > nolibc. We just had to cheat on the pointer test because for NULL nolibc
> > > prints "0x0" while glibc prints "(nil)" so we check 0x1 instead to avoid
> > > this special case.
> > > 
> > > Finally Thomas added a new target to the makefile to ease building the
> > > test against the default libc. This should help detect incompatibilities
> > > when new features are added.
> > > 
> > > I've tested it locally with my libc and against all supported architectures
> > > (userland tests only), and all tests passed.
> > > 
> > > This can be added to your dev tree for 6.5 on top of the previous series.
> > 
> > Looks like some useful code to make testing more comprehensive, thank
> > you both!  Queued and pushed.
> 
> And finally tested:
> 
> Kernel: arch/x86/boot/bzImage is ready  (#7)
> make[1]: Leaving directory '/home/git/linux-build'
> 133 test(s) passed.
> 
>   CC      nolibc-test
> 133 test(s) passed.
> 
> So good agreement between user and kernel, anyway.  Usermode tests
> note that two of the tests are skipped (chroot_root and link_blah).
> Kernelmode tests all say "[OK]".

This is perfect, thank you Paul!

Eventually I'll clean up my test script I use to build and test for all
archs, and commit it to help with such testing.

thanks!
Willy

Re: [PATCH 0/4] tools/nolibc: add testcases for vfprintf

[Paul E. McKenney]

On Tue, Apr 04, 2023 at 10:46:23PM +0200, Willy Tarreau wrote:
> On Tue, Apr 04, 2023 at 01:40:32PM -0700, Paul E. McKenney wrote:
> > On Sun, Apr 02, 2023 at 02:24:04PM -0700, Paul E. McKenney wrote:
> > > On Sun, Apr 02, 2023 at 08:48:02PM +0200, Willy Tarreau wrote:
> > > > Hello Paul,
> > > > 
> > > > Thomas added new tests for vfprintf(), which is a good idea because it
> > > > was a new implementation, we use it a lot in the tests so we'd rather
> > > > make sure it works! This required to implement support for memfd_create()
> > > > that is used to collect the output into a buffer, as well as to complete
> > > > a little bit the minimalistic FILE emulation with fileno(), fdopen(),
> > > > fflush() and fclose(). The result is neat and works equally on glibc and
> > > > nolibc. We just had to cheat on the pointer test because for NULL nolibc
> > > > prints "0x0" while glibc prints "(nil)" so we check 0x1 instead to avoid
> > > > this special case.
> > > > 
> > > > Finally Thomas added a new target to the makefile to ease building the
> > > > test against the default libc. This should help detect incompatibilities
> > > > when new features are added.
> > > > 
> > > > I've tested it locally with my libc and against all supported architectures
> > > > (userland tests only), and all tests passed.
> > > > 
> > > > This can be added to your dev tree for 6.5 on top of the previous series.
> > > 
> > > Looks like some useful code to make testing more comprehensive, thank
> > > you both!  Queued and pushed.
> > 
> > And finally tested:
> > 
> > Kernel: arch/x86/boot/bzImage is ready  (#7)
> > make[1]: Leaving directory '/home/git/linux-build'
> > 133 test(s) passed.
> > 
> >   CC      nolibc-test
> > 133 test(s) passed.
> > 
> > So good agreement between user and kernel, anyway.  Usermode tests
> > note that two of the tests are skipped (chroot_root and link_blah).
> > Kernelmode tests all say "[OK]".
> 
> This is perfect, thank you Paul!
> 
> Eventually I'll clean up my test script I use to build and test for all
> archs, and commit it to help with such testing.

Thank you for confirming the test results!

							Thanx, Paul