Build system mess in stubdom

Build system mess in stubdom

[Andrew Cooper]

Hello,

I'm trying to investigate why stubdom/ is fatally failing now with a
rebuilt ArchLinux container (GCC 14).

It is ultimately:

> ../../../../../newlib-1.16.0/newlib/libc/reent/signalr.c:61:14: error:
> implicit declaration of function ‘kill’; did you mean ‘_kill’?
> [-Wimplicit-function-declaration]
>    61 |   if ((ret = _kill (pid, sig)) == -1 && errno != 0)
>       |              ^~~~~
> make[7]: *** [Makefile:483: lib_a-signalr.o] Error 1

which doesn't make sense, but is a consequence of the ifdefary in
newlib/libc/include/_syslist.h

However, we've got problems ahead of that.

First of all, with:

[user@89aef714763e build]$ ./configure --disable-xen --disable-tools
--disable-docs
<snip>
Will build the following stub domains:
  xenstore-stubdom
  xenstorepvh-stubdom
configure: creating ./config.status
config.status: creating ../config/Stubdom.mk

both a top level `make` and `make stubdom` end up building all of tools,
contrary to comments in the makefile.

`make build-stubdom` does (AFAICT) only build stubdom.

However, building just the xenstore stubdoms recursively builds all of
tools/libs/ even though only some are needed.  This includes libxl which
then recurses further to get tools/libacpi, and libxenguest which
recurses further to get libelf from Xen.

What I can't figure out is why xenstore ends up pulling in all of newlib.

Semi-irrespective, there's no way we can keep on bodging newlib to
compile with newer compilers.  There's a whole bunch of other warnings
(strict-prototypes, dangling-else, maybe-uninitialized, unused-function,
pointer-sign, unused-variable) primed ready to cause breakage in any
environment which makes these error by default.

I'm going to be making ArchLinux non-blocking because it is a rolling
distro, but we also can't do nothing here.

~Andrew

Re: Build system mess in stubdom

[Andrew Cooper]

[Correct Anthony's email]

~Andrew

On 09/07/2024 2:49 pm, Andrew Cooper wrote:
> Hello,
>
> I'm trying to investigate why stubdom/ is fatally failing now with a
> rebuilt ArchLinux container (GCC 14).
>
> It is ultimately:
>
>> ../../../../../newlib-1.16.0/newlib/libc/reent/signalr.c:61:14: error:
>> implicit declaration of function ‘kill’; did you mean ‘_kill’?
>> [-Wimplicit-function-declaration]
>>    61 |   if ((ret = _kill (pid, sig)) == -1 && errno != 0)
>>       |              ^~~~~
>> make[7]: *** [Makefile:483: lib_a-signalr.o] Error 1
> which doesn't make sense, but is a consequence of the ifdefary in
> newlib/libc/include/_syslist.h
>
> However, we've got problems ahead of that.
>
> First of all, with:
>
> [user@89aef714763e build]$ ./configure --disable-xen --disable-tools
> --disable-docs
> <snip>
> Will build the following stub domains:
>   xenstore-stubdom
>   xenstorepvh-stubdom
> configure: creating ./config.status
> config.status: creating ../config/Stubdom.mk
>
> both a top level `make` and `make stubdom` end up building all of tools,
> contrary to comments in the makefile.
>
> `make build-stubdom` does (AFAICT) only build stubdom.
>
> However, building just the xenstore stubdoms recursively builds all of
> tools/libs/ even though only some are needed.  This includes libxl which
> then recurses further to get tools/libacpi, and libxenguest which
> recurses further to get libelf from Xen.
>
> What I can't figure out is why xenstore ends up pulling in all of newlib.
>
> Semi-irrespective, there's no way we can keep on bodging newlib to
> compile with newer compilers.  There's a whole bunch of other warnings
> (strict-prototypes, dangling-else, maybe-uninitialized, unused-function,
> pointer-sign, unused-variable) primed ready to cause breakage in any
> environment which makes these error by default.
>
> I'm going to be making ArchLinux non-blocking because it is a rolling
> distro, but we also can't do nothing here.
>
> ~Andrew

Re: Build system mess in stubdom

[Jan Beulich]

On 09.07.2024 15:49, Andrew Cooper wrote:
> I'm trying to investigate why stubdom/ is fatally failing now with a
> rebuilt ArchLinux container (GCC 14).
> 
> It is ultimately:
> 
>> ../../../../../newlib-1.16.0/newlib/libc/reent/signalr.c:61:14: error:
>> implicit declaration of function ‘kill’; did you mean ‘_kill’?
>> [-Wimplicit-function-declaration]
>>    61 |   if ((ret = _kill (pid, sig)) == -1 && errno != 0)
>>       |              ^~~~~
>> make[7]: *** [Makefile:483: lib_a-signalr.o] Error 1
> 
> which doesn't make sense, but is a consequence of the ifdefary in
> newlib/libc/include/_syslist.h

Charles, who is looking after our Xen packages, had run into exactly this.
His workaround patch (added to the list of patches applied on top of
newlib by stubdom/Makefile) is below, but in the given form I didn't expect
it would be upstreamable. The diagnostics by the compiler may be a little
misleading ...

Jan

--- newlib-1.16.0/newlib/libc/stdlib/wcstoull.c
+++ newlib-1.16.0/newlib/libc/stdlib/wcstoull.c
@@ -127,6 +127,10 @@ PORTABILITY

 #ifndef _REENT_ONLY

+#if __GNUC__ >= 14
+#pragma GCC diagnostic ignored "-Wimplicit-function-declaration"
+#endif
+
 unsigned long long
 _DEFUN (wcstoull, (s, ptr, base),
 	_CONST wchar_t *s _AND
--- newlib-1.16.0/newlib/libc/reent/signalr.c
+++ newlib-1.16.0/newlib/libc/reent/signalr.c
@@ -49,6 +49,10 @@ DESCRIPTION
 	<<errno>>.
 */

+#if __GNUC__ >= 14
+#pragma GCC diagnostic ignored "-Wimplicit-function-declaration"
+#endif
+
 int
 _DEFUN (_kill_r, (ptr, pid, sig),
      struct _reent *ptr _AND
--- newlib-1.16.0/newlib/doc/makedoc.c
+++ newlib-1.16.0/newlib/doc/makedoc.c
@@ -798,6 +798,7 @@ DEFUN( iscommand,(ptr, idx),
 }


+static unsigned int
 DEFUN(copy_past_newline,(ptr, idx, dst),
       string_type *ptr AND
       unsigned int idx AND

Re: Build system mess in stubdom

[Andrew Cooper]

On 09/07/2024 3:55 pm, Jan Beulich wrote:
> On 09.07.2024 15:49, Andrew Cooper wrote:
>> I'm trying to investigate why stubdom/ is fatally failing now with a
>> rebuilt ArchLinux container (GCC 14).
>>
>> It is ultimately:
>>
>>> ../../../../../newlib-1.16.0/newlib/libc/reent/signalr.c:61:14: error:
>>> implicit declaration of function ‘kill’; did you mean ‘_kill’?
>>> [-Wimplicit-function-declaration]
>>>    61 |   if ((ret = _kill (pid, sig)) == -1 && errno != 0)
>>>       |              ^~~~~
>>> make[7]: *** [Makefile:483: lib_a-signalr.o] Error 1
>> which doesn't make sense, but is a consequence of the ifdefary in
>> newlib/libc/include/_syslist.h
> Charles, who is looking after our Xen packages, had run into exactly this.
> His workaround patch (added to the list of patches applied on top of
> newlib by stubdom/Makefile) is below, but in the given form I didn't expect
> it would be upstreamable. The diagnostics by the compiler may be a little
> misleading ...
>
> Jan

Looking through newlib's upstream, I'm going to experiment with updating
to 4.4.0 (Dec 2023).

It's the most recent version, and it's marginally ahead of a very large
number of "make build work with GCC 14" commits.

I feel this is going to be less effort than continuing to duct tape an
obsolete version for newer compilers.

~Andrew

Re: Build system mess in stubdom

[Charles Arnold]

On 7/9/24 8:55 AM, Jan Beulich wrote:
> On 09.07.2024 15:49, Andrew Cooper wrote:
>> I'm trying to investigate why stubdom/ is fatally failing now with a
>> rebuilt ArchLinux container (GCC 14).
>>
>> It is ultimately:
>>
>>> ../../../../../newlib-1.16.0/newlib/libc/reent/signalr.c:61:14: error:
>>> implicit declaration of function ‘kill’; did you mean ‘_kill’?
>>> [-Wimplicit-function-declaration]
>>>     61 |   if ((ret = _kill (pid, sig)) == -1 && errno != 0)
>>>        |              ^~~~~
>>> make[7]: *** [Makefile:483: lib_a-signalr.o] Error 1
>> which doesn't make sense, but is a consequence of the ifdefary in
>> newlib/libc/include/_syslist.h
> Charles, who is looking after our Xen packages, had run into exactly this.
> His workaround patch (added to the list of patches applied on top of
> newlib by stubdom/Makefile) is below, but in the given form I didn't expect
> it would be upstreamable. The diagnostics by the compiler may be a little
> misleading ...

This problem showed up only with gcc14 when it began treating what was 
previously a warning as an error.

Charles

>
> Jan
>
> --- newlib-1.16.0/newlib/libc/stdlib/wcstoull.c
> +++ newlib-1.16.0/newlib/libc/stdlib/wcstoull.c
> @@ -127,6 +127,10 @@ PORTABILITY
>
>   #ifndef _REENT_ONLY
>
> +#if __GNUC__ >= 14
> +#pragma GCC diagnostic ignored "-Wimplicit-function-declaration"
> +#endif
> +
>   unsigned long long
>   _DEFUN (wcstoull, (s, ptr, base),
>   	_CONST wchar_t *s _AND
> --- newlib-1.16.0/newlib/libc/reent/signalr.c
> +++ newlib-1.16.0/newlib/libc/reent/signalr.c
> @@ -49,6 +49,10 @@ DESCRIPTION
>   	<<errno>>.
>   */
>
> +#if __GNUC__ >= 14
> +#pragma GCC diagnostic ignored "-Wimplicit-function-declaration"
> +#endif
> +
>   int
>   _DEFUN (_kill_r, (ptr, pid, sig),
>        struct _reent *ptr _AND
> --- newlib-1.16.0/newlib/doc/makedoc.c
> +++ newlib-1.16.0/newlib/doc/makedoc.c
> @@ -798,6 +798,7 @@ DEFUN( iscommand,(ptr, idx),
>   }
>
>
> +static unsigned int
>   DEFUN(copy_past_newline,(ptr, idx, dst),
>         string_type *ptr AND
>         unsigned int idx AND
>

Re: Build system mess in stubdom

[Anthony PERARD]

On Tue, Jul 09, 2024 at 02:49:57PM +0100, Andrew Cooper wrote:
> Hello,
>
> I'm trying to investigate why stubdom/ is fatally failing now with a
> rebuilt ArchLinux container (GCC 14).
>
> It is ultimately:
>
> > ../../../../../newlib-1.16.0/newlib/libc/reent/signalr.c:61:14: error:
> > implicit declaration of function ‘kill’; did you mean ‘_kill’?
> > [-Wimplicit-function-declaration]
> >    61 |   if ((ret = _kill (pid, sig)) == -1 && errno != 0)
> >       |              ^~~~~
> > make[7]: *** [Makefile:483: lib_a-signalr.o] Error 1
>
> which doesn't make sense, but is a consequence of the ifdefary in
> newlib/libc/include/_syslist.h
>
> However, we've got problems ahead of that.
>
> First of all, with:
>
> [user@89aef714763e build]$ ./configure --disable-xen --disable-tools
> --disable-docs
> <snip>
> Will build the following stub domains:
>   xenstore-stubdom
>   xenstorepvh-stubdom
> configure: creating ./config.status
> config.status: creating ../config/Stubdom.mk
>
> both a top level `make` and `make stubdom` end up building all of tools,
> contrary to comments in the makefile.

:-(, I never noticed that but yeah, that rules is what end up building
the tools:

    install-stubdom: mini-os-dir install-tools

So unless you use one of the build targets, the top makefile end-up
wanting to also install (or dist) the tools. I don't think we can change
that:
    dc497635d93f ("build system: make install-stubdom depend on install-tools again")

> `make build-stubdom` does (AFAICT) only build stubdom.

How do you make that works with `./configure --disable-tools` ? I've got
this:
    $ make build-stubdom
    <snip>
    make -C tools/include build
    ....tools/include/../../tools/Rules.mk:212: *** You have to run ./configure before building or installing the tools.  Stop.
    make: *** [Makefile:44: build-tools-public-headers] Error 2

> However, building just the xenstore stubdoms recursively builds all of
> tools/libs/ even though only some are needed.  This includes libxl which
> then recurses further to get tools/libacpi, and libxenguest which
> recurses further to get libelf from Xen.

libxl? how? Did you run `make -C stubdom xenstore-stubdom`? Or maybe you
used ./configure to select only "xenstore-stubdom"? In that later case
only the build* targets will only build stubdom, the default target as
well as dist* and install* targets will want to "install-tools" as seen
above.

> What I can't figure out is why xenstore ends up pulling in all of newlib.

I think it's because of these in stubdom/Makefile:
    xenstore: $(CROSS_ROOT) xenstore-minios-config.mk
    $(CROSS_ROOT): cross-newlib cross-zlib cross-libpci

> Semi-irrespective, there's no way we can keep on bodging newlib to
> compile with newer compilers.  There's a whole bunch of other warnings
> (strict-prototypes, dangling-else, maybe-uninitialized, unused-function,
> pointer-sign, unused-variable) primed ready to cause breakage in any
> environment which makes these error by default.
>
> I'm going to be making ArchLinux non-blocking because it is a rolling
> distro, but we also can't do nothing here.

I guess we could try to update newlib, 1.16 is from 2007 apparently, and
there's now 4.4 from last year.

Cheers,

--

Anthony Perard | Vates XCP-ng Developer

XCP-ng & Xen Orchestra - Vates solutions

web: https://vates.tech

Re: Build system mess in stubdom

[Andrew Cooper]

On 09/07/2024 4:34 pm, Anthony PERARD wrote:
> On Tue, Jul 09, 2024 at 02:49:57PM +0100, Andrew Cooper wrote:
>> Hello,
>>
>> I'm trying to investigate why stubdom/ is fatally failing now with a
>> rebuilt ArchLinux container (GCC 14).
>>
>> It is ultimately:
>>
>>> ../../../../../newlib-1.16.0/newlib/libc/reent/signalr.c:61:14: error:
>>> implicit declaration of function ‘kill’; did you mean ‘_kill’?
>>> [-Wimplicit-function-declaration]
>>>    61 |   if ((ret = _kill (pid, sig)) == -1 && errno != 0)
>>>       |              ^~~~~
>>> make[7]: *** [Makefile:483: lib_a-signalr.o] Error 1
>> which doesn't make sense, but is a consequence of the ifdefary in
>> newlib/libc/include/_syslist.h
>>
>> However, we've got problems ahead of that.
>>
>> First of all, with:
>>
>> [user@89aef714763e build]$ ./configure --disable-xen --disable-tools
>> --disable-docs
>> <snip>
>> Will build the following stub domains:
>>   xenstore-stubdom
>>   xenstorepvh-stubdom
>> configure: creating ./config.status
>> config.status: creating ../config/Stubdom.mk
>>
>> both a top level `make` and `make stubdom` end up building all of tools,
>> contrary to comments in the makefile.
> :-(, I never noticed that but yeah, that rules is what end up building
> the tools:
>
>     install-stubdom: mini-os-dir install-tools
>
> So unless you use one of the build targets, the top makefile end-up
> wanting to also install (or dist) the tools. I don't think we can change
> that:
>     dc497635d93f ("build system: make install-stubdom depend on install-tools again")

qemu-trad stubdom has been off by default for a bit.  We can probably
delete it entirely before too much longer.

>
>> `make build-stubdom` does (AFAICT) only build stubdom.
> How do you make that works with `./configure --disable-tools` ? I've got
> this:
>     $ make build-stubdom
>     <snip>
>     make -C tools/include build
>     ....tools/include/../../tools/Rules.mk:212: *** You have to run ./configure before building or installing the tools.  Stop.
>     make: *** [Makefile:44: build-tools-public-headers] Error 2

Works for me^W in my random archlinux container...

>
>> However, building just the xenstore stubdoms recursively builds all of
>> tools/libs/ even though only some are needed.  This includes libxl which
>> then recurses further to get tools/libacpi, and libxenguest which
>> recurses further to get libelf from Xen.
> libxl? how? Did you run `make -C stubdom xenstore-stubdom`? Or maybe you
> used ./configure to select only "xenstore-stubdom"? In that later case
> only the build* targets will only build stubdom, the default target as
> well as dist* and install* targets will want to "install-tools" as seen
> above.

again, worked for me like that...

>
>> What I can't figure out is why xenstore ends up pulling in all of newlib.
> I think it's because of these in stubdom/Makefile:
>     xenstore: $(CROSS_ROOT) xenstore-minios-config.mk
>     $(CROSS_ROOT): cross-newlib cross-zlib cross-libpci

:(  More junk that we shouldn't be (re)building just for xenstore.

>> Semi-irrespective, there's no way we can keep on bodging newlib to
>> compile with newer compilers.  There's a whole bunch of other warnings
>> (strict-prototypes, dangling-else, maybe-uninitialized, unused-function,
>> pointer-sign, unused-variable) primed ready to cause breakage in any
>> environment which makes these error by default.
>>
>> I'm going to be making ArchLinux non-blocking because it is a rolling
>> distro, but we also can't do nothing here.
> I guess we could try to update newlib, 1.16 is from 2007 apparently, and
> there's now 4.4 from last year.

Easier said than done.  I've got as far as:

diff --git a/stubdom/configure.ac b/stubdom/configure.ac
index fc736c0387fd..944266bd6dce 100644
--- a/stubdom/configure.ac
+++ b/stubdom/configure.ac
@@ -56,7 +56,7 @@ AX_DEPENDS_PATH_PROG([vtpm], [CMAKE], [cmake])
 # Stubdom libraries version and url setup
 AX_STUBDOM_LIB([ZLIB], [zlib], [1.2.3])
 AX_STUBDOM_LIB([LIBPCI], [libpci], [2.2.9],
[https://mirrors.edge.kernel.org/pub/software/utils/pciutils])
-AX_STUBDOM_LIB([NEWLIB], [newlib], [1.16.0],
[https://sourceware.org/ftp/newlib])
+AX_STUBDOM_LIB([NEWLIB], [newlib], [4.4.0.20231231],
[https://sourceware.org/ftp/newlib])
 AX_STUBDOM_LIB([LWIP], [lwip], [1.3.0],
[https://download.savannah.gnu.org/releases/lwip])
 AX_STUBDOM_LIB([GRUB], [grub], [0.97], [https://alpha.gnu.org/gnu/grub])
 AX_STUBDOM_LIB([GMP], [libgmp], [4.3.2],
[https://gmplib.org/download/gmp/archive])

and deployed onto xenbits/extfiles, but it's stubbornly refusing to compile.

I need to finish some of the other container work more urgently for 4.19.

~Andrew