[PATCH 0/2] nolibc: remove reliance on system headers

[PATCH 0/2] nolibc: remove reliance on system headers

[Thomas Weißschuh]

This was prompted by the discussion about output directory support with
O=.
It seems sometimes we were pulling in system headers making testing
annoying and unreliable.

Signed-off-by: Thomas Weißschuh <linux@weissschuh.net>
---
Thomas Weißschuh (2):
      tools/nolibc: add stdarg.h header
      selftests/nolibc: use -nostdinc for nolibc-test

 tools/include/nolibc/Makefile           |  1 +
 tools/include/nolibc/stdarg.h           | 16 ++++++++++++++++
 tools/testing/selftests/nolibc/Makefile |  2 +-
 3 files changed, 18 insertions(+), 1 deletion(-)
---
base-commit: 556fb7131e03b0283672fb40f6dc2d151752aaa7
change-id: 20230827-nolibc-nostdinc-203908130d67

Best regards,
-- 
Thomas Weißschuh <linux@weissschuh.net>

[PATCH 1/2] tools/nolibc: add stdarg.h header

[Thomas Weißschuh]

This allows nolic to work with `-nostdinc` avoiding any reliance on
system headers.

The implementation has been lifted from musl libc 1.2.4.
There is already an implementation of stdarg.h in include/linux/stdarg.h
but that is GPL licensed and therefore not suitable for nolibc.

The used compiler builtins have been validated to be at least available
since GCC 4.1.2 and clang 3.0.0.

Signed-off-by: Thomas Weißschuh <linux@weissschuh.net>
---
 tools/include/nolibc/Makefile |  1 +
 tools/include/nolibc/stdarg.h | 16 ++++++++++++++++
 2 files changed, 17 insertions(+)

diff --git a/tools/include/nolibc/Makefile b/tools/include/nolibc/Makefile
index 909b6eb500fe..e69c26abe1ea 100644
--- a/tools/include/nolibc/Makefile
+++ b/tools/include/nolibc/Makefile
@@ -34,6 +34,7 @@ all_files := \
 		signal.h \
 		stackprotector.h \
 		std.h \
+		stdarg.h \
 		stdint.h \
 		stdlib.h \
 		string.h \
diff --git a/tools/include/nolibc/stdarg.h b/tools/include/nolibc/stdarg.h
new file mode 100644
index 000000000000..c628b5783da6
--- /dev/null
+++ b/tools/include/nolibc/stdarg.h
@@ -0,0 +1,16 @@
+/* SPDX-License-Identifier: LGPL-2.1 OR MIT */
+/*
+ * Variadic argument support for NOLIBC
+ * Copyright (C) 2005-2020 Rich Felker, et al.
+ */
+
+#ifndef _NOLIBC_STDARG_H
+#define _NOLIBC_STDARG_H
+
+typedef __builtin_va_list va_list;
+#define va_start(v, l)   __builtin_va_start(v, l)
+#define va_end(v)        __builtin_va_end(v)
+#define va_arg(v, l)     __builtin_va_arg(v, l)
+#define va_copy(d, s)    __builtin_va_copy(d, s)
+
+#endif /* _NOLIBC_STDARG_H */

-- 
2.42.0

Re: [PATCH 1/2] tools/nolibc: add stdarg.h header

[Willy Tarreau]

Hi Thomas,

On Sun, Aug 27, 2023 at 10:00:15AM +0200, Thomas Weißschuh wrote:
> This allows nolic to work with `-nostdinc` avoiding any reliance on
> system headers.
> 
> The implementation has been lifted from musl libc 1.2.4.
> There is already an implementation of stdarg.h in include/linux/stdarg.h
> but that is GPL licensed and therefore not suitable for nolibc.

I'm a bit confused because for me, stdarg was normally provided by the
compiler, but I could be mistaken. It's just that it reminds me not so
old memories. Therefore maybe we just need to include or define
"something" to use it.

> +#ifndef _NOLIBC_STDARG_H
> +#define _NOLIBC_STDARG_H
> +
> +typedef __builtin_va_list va_list;
> +#define va_start(v, l)   __builtin_va_start(v, l)
> +#define va_end(v)        __builtin_va_end(v)
> +#define va_arg(v, l)     __builtin_va_arg(v, l)
> +#define va_copy(d, s)    __builtin_va_copy(d, s)
> +
> +#endif /* _NOLIBC_STDARG_H */

Also, regarding the doubt above, I really think these should be guarded
(maybe just use va_start as a hint), because the risk that they come
from libc headers or maybe from the compiler via another include path
is non-negligible.

Just my two cents,
Willy

Re: [PATCH 1/2] tools/nolibc: add stdarg.h header

[Thomas Weißschuh]

Hi Willy!

On 2023-08-29 08:28:27+0200, Willy Tarreau wrote:
> On Sun, Aug 27, 2023 at 10:00:15AM +0200, Thomas Weißschuh wrote:
> > This allows nolic to work with `-nostdinc` avoiding any reliance on
> > system headers.
> > 
> > The implementation has been lifted from musl libc 1.2.4.
> > There is already an implementation of stdarg.h in include/linux/stdarg.h
> > but that is GPL licensed and therefore not suitable for nolibc.
> 
> I'm a bit confused because for me, stdarg was normally provided by the
> compiler, but I could be mistaken. It's just that it reminds me not so
> old memories. Therefore maybe we just need to include or define
> "something" to use it.

It is indeed provided by the compiler.

I could not find anybody doing this differently.
Using builtins seems to me to be the normal way to expose compiler
implementation specifics.

> > +#ifndef _NOLIBC_STDARG_H
> > +#define _NOLIBC_STDARG_H
> > +
> > +typedef __builtin_va_list va_list;
> > +#define va_start(v, l)   __builtin_va_start(v, l)
> > +#define va_end(v)        __builtin_va_end(v)
> > +#define va_arg(v, l)     __builtin_va_arg(v, l)
> > +#define va_copy(d, s)    __builtin_va_copy(d, s)
> > +
> > +#endif /* _NOLIBC_STDARG_H */
> 
> Also, regarding the doubt above, I really think these should be guarded
> (maybe just use va_start as a hint), because the risk that they come
> from libc headers or maybe from the compiler via another include path
> is non-negligible.

I can add a guard.
It would only protect against the case where the other stdarg.h is
loaded first, not if ours is loaded first.

Although these symbols should always only come from some <stdarg.h>
and within a single CU this should always end up being the same file.


Thomas

Re: [PATCH 1/2] tools/nolibc: add stdarg.h header

[Willy Tarreau]

On Tue, Aug 29, 2023 at 11:14:09AM +0200, Thomas Weißschuh wrote:
> Hi Willy!
> 
> On 2023-08-29 08:28:27+0200, Willy Tarreau wrote:
> > On Sun, Aug 27, 2023 at 10:00:15AM +0200, Thomas Weißschuh wrote:
> > > This allows nolic to work with `-nostdinc` avoiding any reliance on
> > > system headers.
> > > 
> > > The implementation has been lifted from musl libc 1.2.4.
> > > There is already an implementation of stdarg.h in include/linux/stdarg.h
> > > but that is GPL licensed and therefore not suitable for nolibc.
> > 
> > I'm a bit confused because for me, stdarg was normally provided by the
> > compiler, but I could be mistaken. It's just that it reminds me not so
> > old memories. Therefore maybe we just need to include or define
> > "something" to use it.
> 
> It is indeed provided by the compiler.

OK. But then, doesn't it mean that if we don't provide our stdarg.h,
the compilers' will be used ? I'm asking because we're already using
va_list and va_args, for example in vfprintf() in stdio.h, which
precisely includes <stdarg.h> so it must indeed come from the compiler.

> I could not find anybody doing this differently.
> Using builtins seems to me to be the normal way to expose compiler
> implementation specifics.

OK but it's already what the compiler does itself in its own stdarg that
is provided. That's why I don't understand what specific case we're trying
to cover here, I feel like we're providing an alternate stdarg in case the
compiler doesn't provide one except that I've not seen a compiler not
provide it (even tcc comes with it), it's like stddef.

Thanks,
Willy

Re: [PATCH 1/2] tools/nolibc: add stdarg.h header

[Thomas Weißschuh]

On 2023-08-29 11:26:19+0200, Willy Tarreau wrote:
> On Tue, Aug 29, 2023 at 11:14:09AM +0200, Thomas Weißschuh wrote:
> > Hi Willy!
> > 
> > On 2023-08-29 08:28:27+0200, Willy Tarreau wrote:
> > > On Sun, Aug 27, 2023 at 10:00:15AM +0200, Thomas Weißschuh wrote:
> > > > This allows nolic to work with `-nostdinc` avoiding any reliance on
> > > > system headers.
> > > > 
> > > > The implementation has been lifted from musl libc 1.2.4.
> > > > There is already an implementation of stdarg.h in include/linux/stdarg.h
> > > > but that is GPL licensed and therefore not suitable for nolibc.
> > > 
> > > I'm a bit confused because for me, stdarg was normally provided by the
> > > compiler, but I could be mistaken. It's just that it reminds me not so
> > > old memories. Therefore maybe we just need to include or define
> > > "something" to use it.
> > 
> > It is indeed provided by the compiler.
> 
> OK. But then, doesn't it mean that if we don't provide our stdarg.h,
> the compilers' will be used ? I'm asking because we're already using
> va_list and va_args, for example in vfprintf() in stdio.h, which
> precisely includes <stdarg.h> so it must indeed come from the compiler.

It will be used *iff* -nostdinc is *not* passed.

I think we need to clarify the definition of the word "provided".
For me it means that the compiler ships an implementation of this header
file in the compiler-specific include directory.

If -nostdinc is passed this include directory is not actually usable.

If a user wants to avoid the implicit usage of any system-provided
headers they need to pass -nostdinc, as far as I know there is no flag
to keep only the compiler-specific include directories.

One usecase is in nolibc-test itself, where Zhangjin ran into weird
and inconsistent behavior of system includes being pulled in.
By using -nostdinc we avoid this.

I can also see this being useful for normal users.

> > I could not find anybody doing this differently.
> > Using builtins seems to me to be the normal way to expose compiler
> > implementation specifics.
> 
> OK but it's already what the compiler does itself in its own stdarg that
> is provided. That's why I don't understand what specific case we're trying
> to cover here, I feel like we're providing an alternate stdarg in case the
> compiler doesn't provide one except that I've not seen a compiler not
> provide it (even tcc comes with it), it's like stddef.

It's all about supporting -nostdinc.


FYI stdint.h is also provided by nolibc, gcc and glibc.

Re: [PATCH 1/2] tools/nolibc: add stdarg.h header

[Willy Tarreau]

On Tue, Aug 29, 2023 at 12:16:23PM +0200, Thomas Weißschuh wrote:
> > OK. But then, doesn't it mean that if we don't provide our stdarg.h,
> > the compilers' will be used ? I'm asking because we're already using
> > va_list and va_args, for example in vfprintf() in stdio.h, which
> > precisely includes <stdarg.h> so it must indeed come from the compiler.
> 
> It will be used *iff* -nostdinc is *not* passed.
> 
> I think we need to clarify the definition of the word "provided".
> For me it means that the compiler ships an implementation of this header
> file in the compiler-specific include directory.
> 
> If -nostdinc is passed this include directory is not actually usable.

OK I understand better now. I thought it was always usable.

> If a user wants to avoid the implicit usage of any system-provided
> headers they need to pass -nostdinc, as far as I know there is no flag
> to keep only the compiler-specific include directories.

So that means we may also have to implement our own stddef.h to move
size_t there, and limits.h and move *MAX there as well if we want to
support this. I'm not necessarily against this, it's just that we need
to be consistent.

Also something is puzzling me. If a normal program builds with -nostdinc,
it means it does *not* want the libc's (nor the compiler's) headers to be
included, probably because it comes with its own. In this case why would
we impose ours ? For example, let's consider this tiny code snippet:

  $ cat arg.c
  #include <stdarg.h>
  va_list blah;

  $ gcc -c arg.c 
  $ gcc -nostdinc -c arg.c
  arg.c:1:20: error: no include path in which to search for stdarg.h
      1 | #include <stdarg.h>
        |                    ^
  arg.c:2:1: error: unknown type name 'va_list'
      2 | va_list blah;
        | ^~~~~~~
  arg.c:1:1: note: 'va_list' is defined in header '<stdarg.h>'; did you forget to '#include <stdarg.h>'?
    +++ |+#include <stdarg.h>
      1 | #include <stdarg.h>
 
You see, that's why I'm finding it confusing that we define headers that
are supposed *not* to be defined with -nostdinc.

I think we need to carefully check what is supposed to be defined and
what not when -nostdinc is used normally so that we defined what programs
expect and not what they expect us *not* to define. Recently we've been
empirically fixing nolibc-test build failures but it's just a test program
that comes with its own biases. Maybe trying to build some portable libs
that use very little from a libc (e.g. xxhash, liblzo etc) could give us
some hints about certain basic assumptions that we do not fulfill.

> One usecase is in nolibc-test itself, where Zhangjin ran into weird
> and inconsistent behavior of system includes being pulled in.
> By using -nostdinc we avoid this.

I see but a normal libc ought not to build with -nostdinc. I mean, we
can define whatever we want once we know why we're doing it, but I think
that as long as we find it confusing between those how are modifying this
code, it will be very difficult to explain correctly to users. We're
definitely missing some design rules I think. Maybe -nostdinc should be
needed only when using -include nolibc.h for example, I don't know, but
I still find that we're papering over a wider problem.

> I can also see this being useful for normal users.

I agree, that's also my concern actually.

> > > I could not find anybody doing this differently.
> > > Using builtins seems to me to be the normal way to expose compiler
> > > implementation specifics.
> > 
> > OK but it's already what the compiler does itself in its own stdarg that
> > is provided. That's why I don't understand what specific case we're trying
> > to cover here, I feel like we're providing an alternate stdarg in case the
> > compiler doesn't provide one except that I've not seen a compiler not
> > provide it (even tcc comes with it), it's like stddef.
> 
> It's all about supporting -nostdinc.

But unless I'm mistaken (and my example above seems to support this),
a normal libc doesn't build with -nostdinc. That's the point I'd like
us to clarify.

> FYI stdint.h is also provided by nolibc, gcc and glibc.

True but that one didn't surprise me because it came with C99 and was
usually shipped by the libc when compilers targetting previous versions
were used, so I didn't see this as a replacement for the compiler's
definition actually.

I don't know what dictates what goes in the compiler and what in the
libc.  I'm fine with having to redefine everything that's missing if
that's needed, but as indicated above, stddef.h and limits.h are
missing despite being quite common.

We have an interesting comment at the top of nolibc.h which says:

 * The available standard (but limited) include files are:
 *   ctype.h, errno.h, signal.h, stdio.h, stdlib.h, string.h, time.h
 *
 * In addition, the following ones are expected to be provided by the compiler:
 *   float.h, stdarg.h, stddef.h
 *
 * The following ones which are part to the C standard are not provided:
 *   assert.h, locale.h, math.h, setjmp.h, limits.h

I think I draw the line based on what my compilers have always provided.
That's definitely something we can redefine (and update the comment),
I'm just seeking consistency, and I think you can understand :-/

Thanks,
Willy

Re: [PATCH 1/2] tools/nolibc: add stdarg.h header

[Thomas Weißschuh]

On 2023-08-29 14:12:45+0200, Willy Tarreau wrote:
> On Tue, Aug 29, 2023 at 12:16:23PM +0200, Thomas Weißschuh wrote:
> > > OK. But then, doesn't it mean that if we don't provide our stdarg.h,
> > > the compilers' will be used ? I'm asking because we're already using
> > > va_list and va_args, for example in vfprintf() in stdio.h, which
> > > precisely includes <stdarg.h> so it must indeed come from the compiler.
> > 
> > It will be used *iff* -nostdinc is *not* passed.
> > 
> > I think we need to clarify the definition of the word "provided".
> > For me it means that the compiler ships an implementation of this header
> > file in the compiler-specific include directory.
> > 
> > If -nostdinc is passed this include directory is not actually usable.
> 
> OK I understand better now. I thought it was always usable.
> 
> > If a user wants to avoid the implicit usage of any system-provided
> > headers they need to pass -nostdinc, as far as I know there is no flag
> > to keep only the compiler-specific include directories.
> 
> So that means we may also have to implement our own stddef.h to move
> size_t there, and limits.h and move *MAX there as well if we want to
> support this. I'm not necessarily against this, it's just that we need
> to be consistent.

We would have to, *iff* the goal is to provide *all* headers in nolibc.
May goal was more limited:
nolibc should be self-contained, it should be able to work at all
with -nostdinc.
If users need more standard headers for their application they can add
those either as shim, custom implementation or from the compiler.

> Also something is puzzling me. If a normal program builds with -nostdinc,
> it means it does *not* want the libc's (nor the compiler's) headers to be
> included, probably because it comes with its own. In this case why would
> we impose ours ? For example, let's consider this tiny code snippet:
> 
>   $ cat arg.c
>   #include <stdarg.h>
>   va_list blah;
> 
>   $ gcc -c arg.c 
>   $ gcc -nostdinc -c arg.c
>   arg.c:1:20: error: no include path in which to search for stdarg.h
>       1 | #include <stdarg.h>
>         |                    ^
>   arg.c:2:1: error: unknown type name 'va_list'
>       2 | va_list blah;
>         | ^~~~~~~
>   arg.c:1:1: note: 'va_list' is defined in header '<stdarg.h>'; did you forget to '#include <stdarg.h>'?
>     +++ |+#include <stdarg.h>
>       1 | #include <stdarg.h>
>  
> You see, that's why I'm finding it confusing that we define headers that
> are supposed *not* to be defined with -nostdinc.

I'm confused.

If the user doesn't want to use nolibc they should not explicitly add it
to the include path.

> I think we need to carefully check what is supposed to be defined and
> what not when -nostdinc is used normally so that we defined what programs
> expect and not what they expect us *not* to define. Recently we've been
> empirically fixing nolibc-test build failures but it's just a test program
> that comes with its own biases. Maybe trying to build some portable libs
> that use very little from a libc (e.g. xxhash, liblzo etc) could give us
> some hints about certain basic assumptions that we do not fulfill.

It makes sense to figure out what is needed by larger projects from a
libc. But it feels to me like a bug vs. feature discussion.

Making larger real-world applications work is a feature while making the
following work is a bugfix:

$ cat nolibc.c
#include "nolibc.h"

int main(void)
{
	return 0;
}

$ gcc -nostdinc -Isysroot/x86/include -c nolibc.c
In file included from sysroot/x86/include/nolibc.h:98,
                 from nolibc-test.c:1:
sysroot/x86/include/sys.h:10:10: fatal error: stdarg.h: No such file or directory
   10 | #include <stdarg.h>
      |          ^~~~~~~~~~

> > One usecase is in nolibc-test itself, where Zhangjin ran into weird
> > and inconsistent behavior of system includes being pulled in.
> > By using -nostdinc we avoid this.
> 
> I see but a normal libc ought not to build with -nostdinc. I mean, we
> can define whatever we want once we know why we're doing it, but I think
> that as long as we find it confusing between those how are modifying this
> code, it will be very difficult to explain correctly to users. We're
> definitely missing some design rules I think. Maybe -nostdinc should be
> needed only when using -include nolibc.h for example, I don't know, but
> I still find that we're papering over a wider problem.
> 
> > I can also see this being useful for normal users.
> 
> I agree, that's also my concern actually.
> 
> > > > I could not find anybody doing this differently.
> > > > Using builtins seems to me to be the normal way to expose compiler
> > > > implementation specifics.
> > > 
> > > OK but it's already what the compiler does itself in its own stdarg that
> > > is provided. That's why I don't understand what specific case we're trying
> > > to cover here, I feel like we're providing an alternate stdarg in case the
> > > compiler doesn't provide one except that I've not seen a compiler not
> > > provide it (even tcc comes with it), it's like stddef.
> > 
> > It's all about supporting -nostdinc.
> 
> But unless I'm mistaken (and my example above seems to support this),
> a normal libc doesn't build with -nostdinc. That's the point I'd like
> us to clarify.

musl:

$ cat /usr/lib/musl/lib/musl-gcc.specs
...
*cc1:
%(cc1_cpu) -nostdinc -isystem /usr/lib/musl/include -isystem include%s
...


dietlibc:

$ cat Makefile
...
DEFAULTCFLAGS=-pipe -nostdinc -D_REENTRANT $(EXTRACFLAGS)
...


klibc re-adds the compilers include path,
This is an alternative we could also use:

$ cat Makefile
...
NOSTDINC_FLAGS := -nostdlib -nostdinc -isystem $(shell $(CC) -print-file-name=include)
...

(these are all I checked)

> > FYI stdint.h is also provided by nolibc, gcc and glibc.
> 
> True but that one didn't surprise me because it came with C99 and was
> usually shipped by the libc when compilers targetting previous versions
> were used, so I didn't see this as a replacement for the compiler's
> definition actually.
> 
> I don't know what dictates what goes in the compiler and what in the
> libc.  I'm fine with having to redefine everything that's missing if
> that's needed, but as indicated above, stddef.h and limits.h are
> missing despite being quite common.

I think it's not really clearly defined what goes where.

There was also a longer discussion on LKML about linux/stdarg.h [0]

The gcc authors argue that Linux should not ship a custom stdarg.h.
But in reality Linux, musl, dietlibc (and probably some more) today are
shipping their own stdarg.h.

> We have an interesting comment at the top of nolibc.h which says:
> 
>  * The available standard (but limited) include files are:
>  *   ctype.h, errno.h, signal.h, stdio.h, stdlib.h, string.h, time.h

This is out of date. It's missing signal.h, stdint.h, unistd.h.

>  *
>  * In addition, the following ones are expected to be provided by the compiler:
>  *   float.h, stdarg.h, stddef.h

What does "expected" mean here?
nolibc itself is perfectly fine without float.h and stddef.h.

>  *
>  * The following ones which are part to the C standard are not provided:
>  *   assert.h, locale.h, math.h, setjmp.h, limits.h

While true, a lot of other headers are also not provided.

> I think I draw the line based on what my compilers have always provided.
> That's definitely something we can redefine (and update the comment),
> I'm just seeking consistency, and I think you can understand :-/

I do understand.

To reiterate it here explicitly, in my opinion it's a worthwhile and
consistent goal to make "nolibc usable standalone with -nostdinc" for
maximal control by the user.

If not, I'd like to use the "-nostdinc -I$(cc -print-file-name=include)"
method to avoid dependencies on system header for nolibc-test
specifically.

Thomas

[0] https://lore.kernel.org/lkml/CAHk-=wgoX0pVqNMMOcrhq=nuOfoZB_3qihyHB3y1S8qo=MDs6w@mail.gmail.com/

Re: [PATCH 1/2] tools/nolibc: add stdarg.h header

[Willy Tarreau]

On Wed, Aug 30, 2023 at 08:21:51AM +0200, Thomas Weißschuh wrote:
> On 2023-08-29 14:12:45+0200, Willy Tarreau wrote:
> > On Tue, Aug 29, 2023 at 12:16:23PM +0200, Thomas Weißschuh wrote:
> > > > OK. But then, doesn't it mean that if we don't provide our stdarg.h,
> > > > the compilers' will be used ? I'm asking because we're already using
> > > > va_list and va_args, for example in vfprintf() in stdio.h, which
> > > > precisely includes <stdarg.h> so it must indeed come from the compiler.
> > > 
> > > It will be used *iff* -nostdinc is *not* passed.
> > > 
> > > I think we need to clarify the definition of the word "provided".
> > > For me it means that the compiler ships an implementation of this header
> > > file in the compiler-specific include directory.
> > > 
> > > If -nostdinc is passed this include directory is not actually usable.
> > 
> > OK I understand better now. I thought it was always usable.
> > 
> > > If a user wants to avoid the implicit usage of any system-provided
> > > headers they need to pass -nostdinc, as far as I know there is no flag
> > > to keep only the compiler-specific include directories.
> > 
> > So that means we may also have to implement our own stddef.h to move
> > size_t there, and limits.h and move *MAX there as well if we want to
> > support this. I'm not necessarily against this, it's just that we need
> > to be consistent.
> 
> We would have to, *iff* the goal is to provide *all* headers in nolibc.

That has never been my goal (especially for those already provided by
the compiler).

> > Also something is puzzling me. If a normal program builds with -nostdinc,
> > it means it does *not* want the libc's (nor the compiler's) headers to be
> > included, probably because it comes with its own. In this case why would
> > we impose ours ? For example, let's consider this tiny code snippet:
> > 
> >   $ cat arg.c
> >   #include <stdarg.h>
> >   va_list blah;
> > 
> >   $ gcc -c arg.c 
> >   $ gcc -nostdinc -c arg.c
> >   arg.c:1:20: error: no include path in which to search for stdarg.h
> >       1 | #include <stdarg.h>
> >         |                    ^
> >   arg.c:2:1: error: unknown type name 'va_list'
> >       2 | va_list blah;
> >         | ^~~~~~~
> >   arg.c:1:1: note: 'va_list' is defined in header '<stdarg.h>'; did you forget to '#include <stdarg.h>'?
> >     +++ |+#include <stdarg.h>
> >       1 | #include <stdarg.h>
> >  
> > You see, that's why I'm finding it confusing that we define headers that
> > are supposed *not* to be defined with -nostdinc.
> 
> I'm confused.
> 
> If the user doesn't want to use nolibc they should not explicitly add it
> to the include path.

I didn't understand that it was what you were seeking, I thought you
wanted to build like above, hence my confusion, see below.

> > I think we need to carefully check what is supposed to be defined and
> > what not when -nostdinc is used normally so that we defined what programs
> > expect and not what they expect us *not* to define. Recently we've been
> > empirically fixing nolibc-test build failures but it's just a test program
> > that comes with its own biases. Maybe trying to build some portable libs
> > that use very little from a libc (e.g. xxhash, liblzo etc) could give us
> > some hints about certain basic assumptions that we do not fulfill.
> 
> It makes sense to figure out what is needed by larger projects from a
> libc. But it feels to me like a bug vs. feature discussion.
> 
> Making larger real-world applications work is a feature while making the
> following work is a bugfix:
> 
> $ cat nolibc.c
> #include "nolibc.h"
> 
> int main(void)
> {
> 	return 0;
> }
> 
> $ gcc -nostdinc -Isysroot/x86/include -c nolibc.c
> In file included from sysroot/x86/include/nolibc.h:98,
>                  from nolibc-test.c:1:
> sysroot/x86/include/sys.h:10:10: fatal error: stdarg.h: No such file or directory
>    10 | #include <stdarg.h>
>       |          ^~~~~~~~~~

This one definitely is a bug, I totally agree. And I didn't understand
this from your initial patch, my understanding was that users would
want to use -nostdinc yet build using regular includes that we'd
provide.

> > > It's all about supporting -nostdinc.

Yes, but "-nostdinc" with "-include nolibc.h". It was probably obvious
to you since you were trying to make it work but I really didn't grasp
it.

> > But unless I'm mistaken (and my example above seems to support this),
> > a normal libc doesn't build with -nostdinc. That's the point I'd like
> > us to clarify.
> 
> musl:
> 
> $ cat /usr/lib/musl/lib/musl-gcc.specs
> ...
> *cc1:
> %(cc1_cpu) -nostdinc -isystem /usr/lib/musl/include -isystem include%s
> ...
> 
> 
> dietlibc:
> 
> $ cat Makefile
> ...
> DEFAULTCFLAGS=-pipe -nostdinc -D_REENTRANT $(EXTRACFLAGS)
> ...

OK.

> klibc re-adds the compilers include path,
> This is an alternative we could also use:
> 
> $ cat Makefile
> ...
> NOSTDINC_FLAGS := -nostdlib -nostdinc -isystem $(shell $(CC) -print-file-name=include)
> ...

Another approach but not easy to pass to end-users.

> There was also a longer discussion on LKML about linux/stdarg.h [0]

Thanks for the link, interesting indeed!

> The gcc authors argue that Linux should not ship a custom stdarg.h.
> But in reality Linux, musl, dietlibc (and probably some more) today are
> shipping their own stdarg.h.

I think the main problem precisely is that -nostdinc excludes both the
system libc's and the compiler's headers (that last point I didn't know).
If there was a standard way to say "no system includes but please keep
the compiler's headers as they're the only exposed interface we have" it
would be much easier. So yes, I understand what you ended up on: you in
fact want to be sure not to inherit from the local system headers and as
a side effect you lose the compiler ones so you need to redefine them.
Then of course that's fine.

> > We have an interesting comment at the top of nolibc.h which says:
> > 
> >  * The available standard (but limited) include files are:
> >  *   ctype.h, errno.h, signal.h, stdio.h, stdlib.h, string.h, time.h
> 
> This is out of date. It's missing signal.h, stdint.h, unistd.h.

Yes very likely, but I found it interesting to find this split that was
done a while ago.

> >  *
> >  * In addition, the following ones are expected to be provided by the compiler:
> >  *   float.h, stdarg.h, stddef.h
> 
> What does "expected" mean here?
> nolibc itself is perfectly fine without float.h and stddef.h.

i.e. "if needed we'll use these ones".

> >  * The following ones which are part to the C standard are not provided:
> >  *   assert.h, locale.h, math.h, setjmp.h, limits.h
> 
> While true, a lot of other headers are also not provided.

Sure, but these were the ones I identified by then.

> > I think I draw the line based on what my compilers have always provided.
> > That's definitely something we can redefine (and update the comment),
> > I'm just seeking consistency, and I think you can understand :-/
> 
> I do understand.
> 
> To reiterate it here explicitly, in my opinion it's a worthwhile and
> consistent goal to make "nolibc usable standalone with -nostdinc" for
> maximal control by the user.

I agree now. I think we need to make it clear that it's for when we're
including the all-in-one "nolibc.h" as an alternative to regular headers.

> If not, I'd like to use the "-nostdinc -I$(cc -print-file-name=include)"
> method to avoid dependencies on system header for nolibc-test
> specifically.

It's a bit ugly and not always easy to stuff into projects. The fact that
nolibc itself isn't self-sustaining anymore with -nostdinc is a concern
and I agree with addressing it like you proposed.

Thanks for the clarification!
Willy

Re: [PATCH 1/2] tools/nolibc: add stdarg.h header

[Willy Tarreau]

On Sun, Aug 27, 2023 at 10:00:15AM +0200, Thomas Weißschuh wrote:
> This allows nolic to work with `-nostdinc` avoiding any reliance on
> system headers.
> 
> The implementation has been lifted from musl libc 1.2.4.
> There is already an implementation of stdarg.h in include/linux/stdarg.h
> but that is GPL licensed and therefore not suitable for nolibc.
> 
> The used compiler builtins have been validated to be at least available
> since GCC 4.1.2 and clang 3.0.0.
> 
> Signed-off-by: Thomas Weißschuh <linux@weissschuh.net>
> ---
>  tools/include/nolibc/Makefile |  1 +
>  tools/include/nolibc/stdarg.h | 16 ++++++++++++++++
>  2 files changed, 17 insertions(+)
> 
> diff --git a/tools/include/nolibc/Makefile b/tools/include/nolibc/Makefile
> index 909b6eb500fe..e69c26abe1ea 100644
> --- a/tools/include/nolibc/Makefile
> +++ b/tools/include/nolibc/Makefile
> @@ -34,6 +34,7 @@ all_files := \
>  		signal.h \
>  		stackprotector.h \
>  		std.h \
> +		stdarg.h \
>  		stdint.h \
>  		stdlib.h \
>  		string.h \
> diff --git a/tools/include/nolibc/stdarg.h b/tools/include/nolibc/stdarg.h
> new file mode 100644
> index 000000000000..c628b5783da6
> --- /dev/null
> +++ b/tools/include/nolibc/stdarg.h
> @@ -0,0 +1,16 @@
> +/* SPDX-License-Identifier: LGPL-2.1 OR MIT */
> +/*
> + * Variadic argument support for NOLIBC
> + * Copyright (C) 2005-2020 Rich Felker, et al.
> + */
> +
> +#ifndef _NOLIBC_STDARG_H
> +#define _NOLIBC_STDARG_H
> +
> +typedef __builtin_va_list va_list;
> +#define va_start(v, l)   __builtin_va_start(v, l)
> +#define va_end(v)        __builtin_va_end(v)
> +#define va_arg(v, l)     __builtin_va_arg(v, l)
> +#define va_copy(d, s)    __builtin_va_copy(d, s)
> +
> +#endif /* _NOLIBC_STDARG_H */

Now with your other explanation I agree, however we need to change:

  #include <stdarg.h>

to

  #include "stdarg.h"

in stdio.h and sys.h so that we always use ours from now on.

Willy

[PATCH 2/2] selftests/nolibc: use -nostdinc for nolibc-test

[Thomas Weißschuh]

Avoid any accidental reliance on system includes.

Signed-off-by: Thomas Weißschuh <linux@weissschuh.net>
---
 tools/testing/selftests/nolibc/Makefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/testing/selftests/nolibc/Makefile b/tools/testing/selftests/nolibc/Makefile
index dfe66776a331..689658f81a19 100644
--- a/tools/testing/selftests/nolibc/Makefile
+++ b/tools/testing/selftests/nolibc/Makefile
@@ -170,7 +170,7 @@ sysroot/$(ARCH)/include:
 ifneq ($(NOLIBC_SYSROOT),0)
 nolibc-test: nolibc-test.c sysroot/$(ARCH)/include
 	$(QUIET_CC)$(CC) $(CFLAGS) $(LDFLAGS) -o $@ \
-	  -nostdlib -static -Isysroot/$(ARCH)/include $< -lgcc
+	  -nostdlib -nostdinc -static -Isysroot/$(ARCH)/include $< -lgcc
 else
 nolibc-test: nolibc-test.c
 	$(QUIET_CC)$(CC) $(CFLAGS) $(LDFLAGS) -o $@ \

-- 
2.42.0