[PATCH 0/3] tools/nolibc: open() cleanups

[PATCH 0/3] tools/nolibc: open() cleanups

[Thomas Weißschuh]

Signed-off-by: Thomas Weißschuh <linux@weissschuh.net>
---
Thomas Weißschuh (3):
      selftests/nolibc: drop unnecessary 'mode' argument to open()
      tools/nolibc: add creat()
      tools/nolibc: simplify mode handling in open() and openat()

 tools/include/nolibc/compiler.h              |  2 ++
 tools/include/nolibc/fcntl.h                 | 42 +++++++++++++++-------------
 tools/testing/selftests/nolibc/nolibc-test.c |  2 +-
 3 files changed, 25 insertions(+), 21 deletions(-)
---
base-commit: e2d3bdc437a3c6fc0fc1cd09244c3686964d769f
change-id: 20260419-nolibc-open-mode-63edcde8e9e3

Best regards,
--  
Thomas Weißschuh <linux@weissschuh.net>

[PATCH 1/3] selftests/nolibc: drop unnecessary 'mode' argument to open()

[Thomas Weißschuh]

The mode is unnecessary here, drop it.

Signed-off-by: Thomas Weißschuh <linux@weissschuh.net>
---
 tools/testing/selftests/nolibc/nolibc-test.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/testing/selftests/nolibc/nolibc-test.c b/tools/testing/selftests/nolibc/nolibc-test.c
index a8e88b5ff20a..36d769736356 100644
--- a/tools/testing/selftests/nolibc/nolibc-test.c
+++ b/tools/testing/selftests/nolibc/nolibc-test.c
@@ -797,7 +797,7 @@ int test_getdents64(const char *dir)
 	int fd, ret;
 	int err;
 
-	ret = fd = open(dir, O_RDONLY | O_DIRECTORY, 0);
+	ret = fd = open(dir, O_RDONLY | O_DIRECTORY);
 	if (ret < 0)
 		return ret;
 

-- 
2.53.0

[PATCH 2/3] tools/nolibc: add creat()

[Thomas Weißschuh]

creat() is a simple wrapper around open().

Signed-off-by: Thomas Weißschuh <linux@weissschuh.net>
---
 tools/include/nolibc/fcntl.h | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/tools/include/nolibc/fcntl.h b/tools/include/nolibc/fcntl.h
index ed2f5553c65a..56650a36f856 100644
--- a/tools/include/nolibc/fcntl.h
+++ b/tools/include/nolibc/fcntl.h
@@ -66,4 +66,14 @@ int open(const char *path, int flags, ...)
 	return __sysret(_sys_open(path, flags, mode));
 }
 
+/*
+ * int creat(const char *path, mode_t mode);
+ */
+
+static __attribute__((unused))
+int creat(const char *path, mode_t mode)
+{
+	return open(path, O_CREAT | O_WRONLY | O_TRUNC, mode);
+}
+
 #endif /* _NOLIBC_FCNTL_H */

-- 
2.53.0

[PATCH 3/3] tools/nolibc: simplify mode handling in open() and openat()

[Thomas Weißschuh]

The current handling of the optional mode arguments using va_list has
some drawbacks. It is hard for the compiler to optimize away and it
needs specific code to handle the O_ flags that need to pass the mode
parameter. Currently that mode parameter is not respected for O_TMPFILE,
which is a bug.

Switch to a macro-based variant which does not generate any additional
code and avoid the explicit handling of 'mode'.
The macros require somewhat recent compiler versions, but users stuck on
old compilers have a trivial workaround by always specifying mode.

Signed-off-by: Thomas Weißschuh <linux@weissschuh.net>

---
It might also possible to use macros similar to syscall() to not require
__VA_OPT__, but those would be fairly ugly.
---
 tools/include/nolibc/compiler.h |  2 ++
 tools/include/nolibc/fcntl.h    | 34 +++++++++++++---------------------
 2 files changed, 15 insertions(+), 21 deletions(-)

diff --git a/tools/include/nolibc/compiler.h b/tools/include/nolibc/compiler.h
index b56570bf9f69..a154aa4e143a 100644
--- a/tools/include/nolibc/compiler.h
+++ b/tools/include/nolibc/compiler.h
@@ -90,4 +90,6 @@
 #  define __nolibc_no_sanitize_undefined
 #endif
 
+#define __nolibc_first_arg(_a1, ...) _a1
+
 #endif /* _NOLIBC_COMPILER_H */
diff --git a/tools/include/nolibc/fcntl.h b/tools/include/nolibc/fcntl.h
index 56650a36f856..89436a72e987 100644
--- a/tools/include/nolibc/fcntl.h
+++ b/tools/include/nolibc/fcntl.h
@@ -25,18 +25,8 @@ int _sys_openat(int dirfd, const char *path, int flags, mode_t mode)
 }
 
 static __attribute__((unused))
-int openat(int dirfd, const char *path, int flags, ...)
+int openat(int dirfd, const char *path, int flags, mode_t mode)
 {
-	mode_t mode = 0;
-
-	if (flags & O_CREAT) {
-		va_list args;
-
-		va_start(args, flags);
-		mode = va_arg(args, mode_t);
-		va_end(args);
-	}
-
 	return __sysret(_sys_openat(dirfd, path, flags, mode));
 }
 
@@ -51,20 +41,22 @@ int _sys_open(const char *path, int flags, mode_t mode)
 }
 
 static __attribute__((unused))
-int open(const char *path, int flags, ...)
+int open(const char *path, int flags, mode_t mode)
 {
-	mode_t mode = 0;
+	return __sysret(_sys_open(path, flags, mode));
+}
 
-	if (flags & O_CREAT) {
-		va_list args;
+#if __nolibc_gnuc_version >= __nolibc_version(8, 0, 0) || \
+	__nolibc_clang_version >= __nolibc_version(12, 0, 0)
 
-		va_start(args, flags);
-		mode = va_arg(args, mode_t);
-		va_end(args);
-	}
+#  define __nolibc_open_mode(...) __nolibc_first_arg(__VA_ARGS__ __VA_OPT__(,) 0)
 
-	return __sysret(_sys_open(path, flags, mode));
-}
+#  define open(path, flags, ...) \
+	  open(path, flags, __nolibc_open_mode(__VA_ARGS__))
+#  define openat(dirfd, path, flags, ...) \
+	  openat(dirfd, path, flags, __nolibc_open_mode(__VA_ARGS__))
+
+#endif
 
 /*
  * int creat(const char *path, mode_t mode);

-- 
2.53.0

Re: [PATCH 1/3] selftests/nolibc: drop unnecessary 'mode' argument to open()

[Willy Tarreau]

On Sun, Apr 19, 2026 at 05:29:03PM +0200, Thomas Weißschuh wrote:
> The mode is unnecessary here, drop it.

Indeed and it's even desirable *not* to have it since applications will
generally not have it either and we'd prefer to detect bugs that affect
them.

> Signed-off-by: Thomas Weißschuh <linux@weissschuh.net>

Acked-by: Willy Tarreau <w@1wt.eu>

willy

Re: [PATCH 3/3] tools/nolibc: simplify mode handling in open() and openat()

[Willy Tarreau]

On Sun, Apr 19, 2026 at 05:29:05PM +0200, Thomas Weißschuh wrote:
> The current handling of the optional mode arguments using va_list has
> some drawbacks. It is hard for the compiler to optimize away and it
> needs specific code to handle the O_ flags that need to pass the mode
> parameter. Currently that mode parameter is not respected for O_TMPFILE,
> which is a bug.
> 
> Switch to a macro-based variant which does not generate any additional
> code and avoid the explicit handling of 'mode'.
> The macros require somewhat recent compiler versions, but users stuck on
> old compilers have a trivial workaround by always specifying mode.
> 
> Signed-off-by: Thomas Weißschuh <linux@weissschuh.net>
> 
> ---
> It might also possible to use macros similar to syscall() to not require
> __VA_OPT__, but those would be fairly ugly.
> ---
>  tools/include/nolibc/compiler.h |  2 ++
>  tools/include/nolibc/fcntl.h    | 34 +++++++++++++---------------------
>  2 files changed, 15 insertions(+), 21 deletions(-)
> 
> diff --git a/tools/include/nolibc/compiler.h b/tools/include/nolibc/compiler.h
> index b56570bf9f69..a154aa4e143a 100644
> --- a/tools/include/nolibc/compiler.h
> +++ b/tools/include/nolibc/compiler.h
> @@ -90,4 +90,6 @@
>  #  define __nolibc_no_sanitize_undefined
>  #endif
>  
> +#define __nolibc_first_arg(_a1, ...) _a1
> +
>  #endif /* _NOLIBC_COMPILER_H */
> diff --git a/tools/include/nolibc/fcntl.h b/tools/include/nolibc/fcntl.h
> index 56650a36f856..89436a72e987 100644
> --- a/tools/include/nolibc/fcntl.h
> +++ b/tools/include/nolibc/fcntl.h
> @@ -25,18 +25,8 @@ int _sys_openat(int dirfd, const char *path, int flags, mode_t mode)
>  }
>  
>  static __attribute__((unused))
> -int openat(int dirfd, const char *path, int flags, ...)
> +int openat(int dirfd, const char *path, int flags, mode_t mode)
>  {
> -	mode_t mode = 0;
> -
> -	if (flags & O_CREAT) {
> -		va_list args;
> -
> -		va_start(args, flags);
> -		mode = va_arg(args, mode_t);
> -		va_end(args);
> -	}
> -
>  	return __sysret(_sys_openat(dirfd, path, flags, mode));
>  }
>  
> @@ -51,20 +41,22 @@ int _sys_open(const char *path, int flags, mode_t mode)
>  }
>  
>  static __attribute__((unused))
> -int open(const char *path, int flags, ...)
> +int open(const char *path, int flags, mode_t mode)
>  {
> -	mode_t mode = 0;
> +	return __sysret(_sys_open(path, flags, mode));
> +}
>  
> -	if (flags & O_CREAT) {
> -		va_list args;
> +#if __nolibc_gnuc_version >= __nolibc_version(8, 0, 0) || \
> +	__nolibc_clang_version >= __nolibc_version(12, 0, 0)
>  
> -		va_start(args, flags);
> -		mode = va_arg(args, mode_t);
> -		va_end(args);
> -	}
> +#  define __nolibc_open_mode(...) __nolibc_first_arg(__VA_ARGS__ __VA_OPT__(,) 0)
>  
> -	return __sysret(_sys_open(path, flags, mode));
> -}
> +#  define open(path, flags, ...) \
> +	  open(path, flags, __nolibc_open_mode(__VA_ARGS__))
> +#  define openat(dirfd, path, flags, ...) \
> +	  openat(dirfd, path, flags, __nolibc_open_mode(__VA_ARGS__))
> +
> +#endif
>  
>  /*
>   * int creat(const char *path, mode_t mode);

I'm really confused here. It looks to me like we first define the open()
function then define it as a macro that will override it for recent
compilers. It doesn't make the code easy to follow, and in addition I'm
really seeing this as a step backwards, for having had to systematically
modify existing code to append ',0' to all open() calls in order to build
with nolibc years ago (hence the leftover that you spotted in the test).

It's not clear to me what was the problem you faced with O_TMPFILE in the
first place. If it's just a matter of making the 3rd arg optional and not
depend on a valist, we can instead define open() as a macro that always
passes the 3rd arg as zero when not defined. When I need to handle varargs
in macros, I do it like this:

  #define _OPT_ARG(a0, a1, ...) a1
  #define open(path, flags, mode...) _open(path, flags, _OPT_ARG(0, ##mode, 0))

  int _open(const char *path, int flags, mode_t mode)
  {
     ...
  }

The macro passes the "mode" argument to _open() when it's present, and when
it's absent, it passes the next one in the _OPT_ARG() macro, which is zero.

E.g.:

  #include <stdio.h>
  
  #define _OPT_ARG(a0, a1, ...) a1
  #define open(path, flags, mode...) _open(path, flags, _OPT_ARG(0, ##mode, 0))
  
  int _open(const char *path, int flags, int mode)
  {
     return printf("path=%s flags=%d mode=%d\n", path, flags, mode);
  }
  
  int main(void)
  {
          open("file1", 12);
          open("file2", 34, 56);
          return 0;
  }

  $ ./a.out
  path=file1 flags=12 mode=0
  path=file2 flags=34 mode=56

And this one works even on very old compilers (gcc-3.4 successfully tested).

Willy

Re: [PATCH 2/3] tools/nolibc: add creat()

[Willy Tarreau]

On Sun, Apr 19, 2026 at 05:29:04PM +0200, Thomas Weißschuh wrote:
> creat() is a simple wrapper around open().
> 
> Signed-off-by: Thomas Weißschuh <linux@weissschuh.net>

Acked-by: Willy Tarreau <w@1wt.eu>

Willy

Re: [PATCH 3/3] tools/nolibc: simplify mode handling in open() and openat()

[Thomas Weißschuh]

On 2026-04-19 18:08:07+0200, Willy Tarreau wrote:
> On Sun, Apr 19, 2026 at 05:29:05PM +0200, Thomas Weißschuh wrote:
> > The current handling of the optional mode arguments using va_list has
> > some drawbacks. It is hard for the compiler to optimize away and it
> > needs specific code to handle the O_ flags that need to pass the mode
> > parameter. Currently that mode parameter is not respected for O_TMPFILE,
> > which is a bug.
> > 
> > Switch to a macro-based variant which does not generate any additional
> > code and avoid the explicit handling of 'mode'.
> > The macros require somewhat recent compiler versions, but users stuck on
> > old compilers have a trivial workaround by always specifying mode.
> > 
> > Signed-off-by: Thomas Weißschuh <linux@weissschuh.net>
> > 
> > ---
> > It might also possible to use macros similar to syscall() to not require
> > __VA_OPT__, but those would be fairly ugly.
> > ---
> >  tools/include/nolibc/compiler.h |  2 ++
> >  tools/include/nolibc/fcntl.h    | 34 +++++++++++++---------------------
> >  2 files changed, 15 insertions(+), 21 deletions(-)
> > 
> > diff --git a/tools/include/nolibc/compiler.h b/tools/include/nolibc/compiler.h
> > index b56570bf9f69..a154aa4e143a 100644
> > --- a/tools/include/nolibc/compiler.h
> > +++ b/tools/include/nolibc/compiler.h
> > @@ -90,4 +90,6 @@
> >  #  define __nolibc_no_sanitize_undefined
> >  #endif
> >  
> > +#define __nolibc_first_arg(_a1, ...) _a1
> > +
> >  #endif /* _NOLIBC_COMPILER_H */
> > diff --git a/tools/include/nolibc/fcntl.h b/tools/include/nolibc/fcntl.h
> > index 56650a36f856..89436a72e987 100644
> > --- a/tools/include/nolibc/fcntl.h
> > +++ b/tools/include/nolibc/fcntl.h
> > @@ -25,18 +25,8 @@ int _sys_openat(int dirfd, const char *path, int flags, mode_t mode)
> >  }
> >  
> >  static __attribute__((unused))
> > -int openat(int dirfd, const char *path, int flags, ...)
> > +int openat(int dirfd, const char *path, int flags, mode_t mode)
> >  {
> > -	mode_t mode = 0;
> > -
> > -	if (flags & O_CREAT) {
> > -		va_list args;
> > -
> > -		va_start(args, flags);
> > -		mode = va_arg(args, mode_t);
> > -		va_end(args);
> > -	}
> > -
> >  	return __sysret(_sys_openat(dirfd, path, flags, mode));
> >  }
> >  
> > @@ -51,20 +41,22 @@ int _sys_open(const char *path, int flags, mode_t mode)
> >  }
> >  
> >  static __attribute__((unused))
> > -int open(const char *path, int flags, ...)
> > +int open(const char *path, int flags, mode_t mode)
> >  {
> > -	mode_t mode = 0;
> > +	return __sysret(_sys_open(path, flags, mode));
> > +}
> >  
> > -	if (flags & O_CREAT) {
> > -		va_list args;
> > +#if __nolibc_gnuc_version >= __nolibc_version(8, 0, 0) || \
> > +	__nolibc_clang_version >= __nolibc_version(12, 0, 0)
> >  
> > -		va_start(args, flags);
> > -		mode = va_arg(args, mode_t);
> > -		va_end(args);
> > -	}
> > +#  define __nolibc_open_mode(...) __nolibc_first_arg(__VA_ARGS__ __VA_OPT__(,) 0)
> >  
> > -	return __sysret(_sys_open(path, flags, mode));
> > -}
> > +#  define open(path, flags, ...) \
> > +	  open(path, flags, __nolibc_open_mode(__VA_ARGS__))
> > +#  define openat(dirfd, path, flags, ...) \
> > +	  openat(dirfd, path, flags, __nolibc_open_mode(__VA_ARGS__))
> > +
> > +#endif
> >  
> >  /*
> >   * int creat(const char *path, mode_t mode);
> 
> I'm really confused here. It looks to me like we first define the open()
> function then define it as a macro that will override it for recent
> compilers. It doesn't make the code easy to follow, and in addition I'm
> really seeing this as a step backwards, for having had to systematically
> modify existing code to append ',0' to all open() calls in order to build
> with nolibc years ago (hence the leftover that you spotted in the test).

Why do you see it as a step backwards? The argument is still optional.
As seen by the usage in the selftest.

> It's not clear to me what was the problem you faced with O_TMPFILE in the
> first place. If it's just a matter of making the 3rd arg optional and not
> depend on a valist, we can instead define open() as a macro that always
> passes the 3rd arg as zero when not defined. When I need to handle varargs
> in macros, I do it like this:

O_TMPFILE is required to respect the mode argument. This did not happen
in nolibc. With the current structure we would need to add a check for
flags & O_TMPFILE == O_TMPFILE, as O_TMPFILE is a bitmask, creating even
more code. Maybe future O_ flags would also require special handling.
With the optional argument handling we don't have to care about that at
all. It would be the users responsibility to pass the correct mode.

>   #define _OPT_ARG(a0, a1, ...) a1
>   #define open(path, flags, mode...) _open(path, flags, _OPT_ARG(0, ##mode, 0))
> 
>   int _open(const char *path, int flags, mode_t mode)
>   {
>      ...
>   }
> 
> The macro passes the "mode" argument to _open() when it's present, and when
> it's absent, it passes the next one in the _OPT_ARG() macro, which is zero.
> 
> E.g.:
> 
>   #include <stdio.h>
>   
>   #define _OPT_ARG(a0, a1, ...) a1
>   #define open(path, flags, mode...) _open(path, flags, _OPT_ARG(0, ##mode, 0))
>   
>   int _open(const char *path, int flags, int mode)
>   {
>      return printf("path=%s flags=%d mode=%d\n", path, flags, mode);
>   }

This is exactly what I tried to do. But ,## only works for the
non-first argument and I didn't think of adding a dummy argument, so I
ended up with __VA_OPT__.

As open() is defined to be a function, I used the same name for both the
vararg macro and the underlying function. I slightly prefer that, but if
you object, let's rename it to _open().

>   int main(void)
>   {
>           open("file1", 12);
>           open("file2", 34, 56);
>           return 0;
>   }
> 
>   $ ./a.out
>   path=file1 flags=12 mode=0
>   path=file2 flags=34 mode=56
> 
> And this one works even on very old compilers (gcc-3.4 successfully tested).

Let's use that.


Thomas