* Meson can't recover from deletion of generated QAPI file(s)
@ 2020-09-09 12:37 Markus Armbruster
2020-09-09 13:06 ` Philippe Mathieu-Daudé
` (2 more replies)
0 siblings, 3 replies; 11+ messages in thread
From: Markus Armbruster @ 2020-09-09 12:37 UTC (permalink / raw)
To: Paolo Bonzini, Marc-André Lureau; +Cc: qemu-devel
Watch this:
$ rm qapi/qapi-types-error.h
$ make
Generating qemu-version.h with a meson_exe.py custom command
Compiling C object qom/libqom.fa.p/object_interfaces.c.o
In file included from ../qom/object_interfaces.c:4:
/work/armbru/qemu/include/qapi/error.h:275:10: fatal error: qapi/qapi-types-error.h: No such file or directory
275 | #include "qapi/qapi-types-error.h"
| ^~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
make: *** [Makefile.ninja:348: qom/libqom.fa.p/object_interfaces.c.o] Error 1
To recover, I have to run qapi-gen.py manually and exactly right, or
blow away the build tree and start over.
The old build system did what a build system should: it remade the files
that are missing or out of date.
I'm still too clueless about Meson to debug this, but I hope I can learn
from watching you fix it.
^ permalink raw reply [flat|nested] 11+ messages in thread* Re: Meson can't recover from deletion of generated QAPI file(s) 2020-09-09 12:37 Meson can't recover from deletion of generated QAPI file(s) Markus Armbruster @ 2020-09-09 13:06 ` Philippe Mathieu-Daudé 2020-09-09 13:35 ` Claudio Fontana 2020-09-09 14:00 ` Marc-André Lureau 2020-09-09 14:34 ` Paolo Bonzini 2 siblings, 1 reply; 11+ messages in thread From: Philippe Mathieu-Daudé @ 2020-09-09 13:06 UTC (permalink / raw) To: Markus Armbruster, Paolo Bonzini, Marc-André Lureau Cc: qemu-devel, Claudio Fontana On 9/9/20 2:37 PM, Markus Armbruster wrote: > Watch this: > > $ rm qapi/qapi-types-error.h > $ make > Generating qemu-version.h with a meson_exe.py custom command > Compiling C object qom/libqom.fa.p/object_interfaces.c.o > In file included from ../qom/object_interfaces.c:4: > /work/armbru/qemu/include/qapi/error.h:275:10: fatal error: qapi/qapi-types-error.h: No such file or directory > 275 | #include "qapi/qapi-types-error.h" > | ^~~~~~~~~~~~~~~~~~~~~~~~~ > compilation terminated. > make: *** [Makefile.ninja:348: qom/libqom.fa.p/object_interfaces.c.o] Error 1 > > To recover, I have to run qapi-gen.py manually and exactly right, or > blow away the build tree and start over. > > The old build system did what a build system should: it remade the files > that are missing or out of date. > > I'm still too clueless about Meson to debug this, but I hope I can learn > from watching you fix it. Maybe related: https://www.mail-archive.com/qemu-devel@nongnu.org/msg736135.html and later: https://www.mail-archive.com/qemu-devel@nongnu.org/msg738777.html ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Meson can't recover from deletion of generated QAPI file(s) 2020-09-09 13:06 ` Philippe Mathieu-Daudé @ 2020-09-09 13:35 ` Claudio Fontana 2020-09-09 14:35 ` Paolo Bonzini 0 siblings, 1 reply; 11+ messages in thread From: Claudio Fontana @ 2020-09-09 13:35 UTC (permalink / raw) To: Philippe Mathieu-Daudé, Markus Armbruster, Paolo Bonzini, Marc-André Lureau Cc: qemu-devel On 9/9/20 3:06 PM, Philippe Mathieu-Daudé wrote: > On 9/9/20 2:37 PM, Markus Armbruster wrote: >> Watch this: >> >> $ rm qapi/qapi-types-error.h >> $ make >> Generating qemu-version.h with a meson_exe.py custom command >> Compiling C object qom/libqom.fa.p/object_interfaces.c.o >> In file included from ../qom/object_interfaces.c:4: >> /work/armbru/qemu/include/qapi/error.h:275:10: fatal error: qapi/qapi-types-error.h: No such file or directory >> 275 | #include "qapi/qapi-types-error.h" >> | ^~~~~~~~~~~~~~~~~~~~~~~~~ >> compilation terminated. >> make: *** [Makefile.ninja:348: qom/libqom.fa.p/object_interfaces.c.o] Error 1 >> >> To recover, I have to run qapi-gen.py manually and exactly right, or >> blow away the build tree and start over. >> >> The old build system did what a build system should: it remade the files >> that are missing or out of date. >> >> I'm still too clueless about Meson to debug this, but I hope I can learn >> from watching you fix it. > > Maybe related: > https://www.mail-archive.com/qemu-devel@nongnu.org/msg736135.html > and later: > https://www.mail-archive.com/qemu-devel@nongnu.org/msg738777.html > My use case is this: my simple build script uses git rebase -x to run a build for each commit of a series, and fails if at any point one of the commits breaks the build or tests. FILE ~/bin/check-build.sh: #! /bin/bash git rebase $1 -x build.sh FILE ~/bin/build.sh: #! /bin/bash set -x set -e rm -fr build-tcg mkdir build-tcg cd build-tcg ../configure --enable-tcg --disable-kvm --disable-hax make -j120 make -j120 check cd .. rm -fr build-nontcg mkdir build-nontcg cd build-nontcg ../configure --disable-tcg --enable-kvm --enable-hax make -j120 make -j120 check cd .. rm -fr build-all mkdir build-all cd build-all ../configure --enable-tcg --enable-kvm --enable-hax make -j120 make -j120 check cd .. -------------------- In case it helps with reproducing the problem. Ciao, Claudio ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Meson can't recover from deletion of generated QAPI file(s) 2020-09-09 13:35 ` Claudio Fontana @ 2020-09-09 14:35 ` Paolo Bonzini 2020-09-09 14:36 ` Claudio Fontana 0 siblings, 1 reply; 11+ messages in thread From: Paolo Bonzini @ 2020-09-09 14:35 UTC (permalink / raw) To: Claudio Fontana, Philippe Mathieu-Daudé, Markus Armbruster, Marc-André Lureau Cc: qemu-devel On 09/09/20 15:35, Claudio Fontana wrote: > On 9/9/20 3:06 PM, Philippe Mathieu-Daudé wrote: >> Maybe related: >> https://www.mail-archive.com/qemu-devel@nongnu.org/msg736135.html >> and later: >> https://www.mail-archive.com/qemu-devel@nongnu.org/msg738777.html >> > > My use case is this: Hi Claudio, unlike Markus's, your issue is a genuine meson.build bug. It should be fixed like so: diff --git a/tests/meson.build b/tests/meson.build index 998e4c48f9..721641afbb 100644 --- a/tests/meson.build +++ b/tests/meson.build @@ -223,7 +223,7 @@ foreach test_name, extra: tests src += test_ss.all_sources() deps += test_ss.all_dependencies() endif - exe = executable(test_name, src, dependencies: deps) + exe = executable(test_name, src, genh, dependencies: deps) test(test_name, exe, depends: test_deps.get(test_name, []), Paolo ^ permalink raw reply related [flat|nested] 11+ messages in thread
* Re: Meson can't recover from deletion of generated QAPI file(s) 2020-09-09 14:35 ` Paolo Bonzini @ 2020-09-09 14:36 ` Claudio Fontana 2020-09-09 15:05 ` Claudio Fontana 0 siblings, 1 reply; 11+ messages in thread From: Claudio Fontana @ 2020-09-09 14:36 UTC (permalink / raw) To: Paolo Bonzini, Philippe Mathieu-Daudé, Markus Armbruster, Marc-André Lureau Cc: qemu-devel On 9/9/20 4:35 PM, Paolo Bonzini wrote: > On 09/09/20 15:35, Claudio Fontana wrote: >> On 9/9/20 3:06 PM, Philippe Mathieu-Daudé wrote: >>> Maybe related: >>> https://www.mail-archive.com/qemu-devel@nongnu.org/msg736135.html >>> and later: >>> https://www.mail-archive.com/qemu-devel@nongnu.org/msg738777.html >>> >> >> My use case is this: > > Hi Claudio, > > unlike Markus's, your issue is a genuine meson.build bug. It should be > fixed like so: > > diff --git a/tests/meson.build b/tests/meson.build > index 998e4c48f9..721641afbb 100644 > --- a/tests/meson.build > +++ b/tests/meson.build > @@ -223,7 +223,7 @@ foreach test_name, extra: tests > src += test_ss.all_sources() > deps += test_ss.all_dependencies() > endif > - exe = executable(test_name, src, dependencies: deps) > + exe = executable(test_name, src, genh, dependencies: deps) > > test(test_name, exe, > depends: test_deps.get(test_name, []), > > > Paolo > Will test right away, thanks! C ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Meson can't recover from deletion of generated QAPI file(s) 2020-09-09 14:36 ` Claudio Fontana @ 2020-09-09 15:05 ` Claudio Fontana 2020-09-09 15:10 ` Paolo Bonzini 0 siblings, 1 reply; 11+ messages in thread From: Claudio Fontana @ 2020-09-09 15:05 UTC (permalink / raw) To: Paolo Bonzini, Marc-André Lureau Cc: Philippe Mathieu-Daudé, Markus Armbruster, qemu-devel On 9/9/20 4:36 PM, Claudio Fontana wrote: > On 9/9/20 4:35 PM, Paolo Bonzini wrote: >> On 09/09/20 15:35, Claudio Fontana wrote: >>> On 9/9/20 3:06 PM, Philippe Mathieu-Daudé wrote: >>>> Maybe related: >>>> https://www.mail-archive.com/qemu-devel@nongnu.org/msg736135.html >>>> and later: >>>> https://www.mail-archive.com/qemu-devel@nongnu.org/msg738777.html >>>> >>> >>> My use case is this: >> >> Hi Claudio, >> >> unlike Markus's, your issue is a genuine meson.build bug. It should be >> fixed like so: >> >> diff --git a/tests/meson.build b/tests/meson.build >> index 998e4c48f9..721641afbb 100644 >> --- a/tests/meson.build >> +++ b/tests/meson.build >> @@ -223,7 +223,7 @@ foreach test_name, extra: tests >> src += test_ss.all_sources() >> deps += test_ss.all_dependencies() >> endif >> - exe = executable(test_name, src, dependencies: deps) >> + exe = executable(test_name, src, genh, dependencies: deps) >> >> test(test_name, exe, >> depends: test_deps.get(test_name, []), >> >> >> Paolo >> > > Will test right away, thanks! > > C > Hmm, I still encounter problems: Passed all 120 iotests + cd .. + rm -fr build-nontcg + mkdir build-nontcg + cd build-nontcg + ../configure --disable-tcg --enable-kvm --enable-hax cross containers no NOTE: guest cross-compilers enabled: cc The Meson build system Version: 0.55.1 Source dir: /dev/shm/cfontana/qemu Build dir: /dev/shm/cfontana/qemu/build-nontcg Build type: native build Project name: qemu Project version: 5.1.50 C compiler for the host machine: cc (gcc 7.5.0 "cc (SUSE Linux) 7.5.0") C linker for the host machine: cc ld.bfd 2.34.0.20200325-386 Host machine cpu family: x86_64 Host machine cpu: x86_64 ../meson.build:10: WARNING: Module unstable-keyval has no backwards or forwards compatibility and might not exist in future releases. Program sh found: YES Program python3 found: YES (/usr/bin/python3) Configuring ninjatool using configuration C++ compiler for the host machine: c++ (gcc 7.5.0 "c++ (SUSE Linux) 7.5.0") C++ linker for the host machine: c++ ld.bfd 2.34.0.20200325-386 Library m found: YES Library util found: YES Found pkg-config: /usr/bin/pkg-config (0.29.2) Run-time dependency pixman-1 found: YES 0.34.0 Library pam found: YES Library aio found: YES Run-time dependency zlib found: YES 1.2.11 Run-time dependency xkbcommon found: NO (tried pkgconfig) Library rt found: YES sdl2-config found: NO Run-time dependency sdl2 found: NO (tried pkgconfig and config-tool) Run-time dependency libpng found: NO (tried pkgconfig) Has header "jpeglib.h" : NO Has header "sasl/sasl.h" : YES Library sasl2 found: YES Run-time dependency u2f-emu found: NO (tried pkgconfig) Run-time dependency libkeyutils found: NO (tried pkgconfig) Checking for function "gettid" : NO Configuring config-host.h using configuration Program scripts/minikconf.py found: YES Configuring i386-softmmu-config-target.h using configuration Configuring i386-softmmu-config-devices.mak with command Reading depfile: /dev/shm/cfontana/qemu/build-nontcg/meson-private/i386-softmmu-config-devices.mak.d Configuring i386-softmmu-config-devices.h using configuration Configuring x86_64-softmmu-config-target.h using configuration Configuring x86_64-softmmu-config-devices.mak with command Reading depfile: /dev/shm/cfontana/qemu/build-nontcg/meson-private/x86_64-softmmu-config-devices.mak.d Configuring x86_64-softmmu-config-devices.h using configuration Program scripts/hxtool found: YES Program scripts/shaderinclude.pl found: YES Program scripts/qapi-gen.py found: YES Program scripts/qemu-version.sh found: YES Run-time dependency threads found: YES Program keycodemapdb/tools/keymap-gen found: YES Program scripts/decodetree.py found: YES Program ../scripts/modules/module_block.py found: YES Program nm found: YES Program scripts/undefsym.py found: YES Program scripts/feature_to_c.sh found: YES Program bzip2 found: YES Configuring 50-edk2-i386-secure.json using configuration Configuring 50-edk2-x86_64-secure.json using configuration Configuring 60-edk2-aarch64.json using configuration Configuring 60-edk2-arm.json using configuration Configuring 60-edk2-i386.json using configuration Configuring 60-edk2-x86_64.json using configuration Program qemu-keymap found: YES Program python3 found: YES (/usr/bin/python3) Program diff found: YES Program dbus-daemon found: YES Program initrd-stress.sh found: YES Build targets in project: 410 qemu 5.1.50 Install prefix: /usr/local BIOS directory: /usr/local/share/qemu firmware path: /usr/local/share/qemu-firmware binary directory: /usr/local/bin library directory: /usr/local/lib module directory: /usr/local/lib/qemu libexec directory: /usr/local/libexec include directory: /usr/local/include config directory: /usr/local/etc local state directory: /usr/local/var Manual directory: share/man Doc directory: /usr/local/share/doc Build directory: /dev/shm/cfontana/qemu/build-nontcg Source path: /dev/shm/cfontana/qemu GIT binary: git GIT submodules: ui/keycodemapdb tests/fp/berkeley-testfloat-3 tests/fp/berkeley-softfloat-3 meson capstone slirp C compiler: cc Host C compiler: cc C++ compiler: c++ ARFLAGS: rv CFLAGS: -O2 -g -fPIE -DPIE -std=gnu99 -Wall QEMU_CFLAGS: -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -m64 -mcx16 -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -Wstrict-prototypes -Wredundant-decls -Wundef -Wwrite-strings -Wmissing-prototypes -fno-strict-aliasing -fno-common -fwrapv -Wold-style-declaration -Wold-style-definition -Wtype-limits -Wformat-security -Wformat-y2k -Winit-self -Wignored-qualifiers -Wempty-body -Wnested-externs -Wendif-labels -Wexpansion-to-defined -Wno-missing-include-dirs -Wno-shift-negative-value -Wno-psabi -fstack-protector-strong QEMU_LDFLAGS: -Wl,--warn-common -Wl,-z,relro -Wl,-z,now -m64 -fstack-protector-strong make: make python: /usr/bin/python3 (version: 3.6) sphinx-build: genisoimage: /usr/bin/mkisofs slirp support: YES smbd: "/usr/sbin/smbd" module support: NO host CPU: x86_64 host endianness: little target list: i386-softmmu x86_64-softmmu gprof enabled: NO sparse enabled: NO strip binaries: YES profiler: NO static build: YES SDL support: NO SDL image support: NO GTK support: NO GTK GL support: NO pixman: YES VTE support: NO TLS priority: "NORMAL" GNUTLS support: NO libgcrypt: NO nettle: NO libtasn1: NO PAM: YES iconv support: YES curses support: YES virgl support: NO curl support: NO mingw32 support: NO Audio drivers: oss Block whitelist (rw): Block whitelist (ro): VirtFS support: NO Multipath support: NO VNC support: YES VNC SASL support: YES VNC JPEG support: NO VNC PNG support: NO xen support: NO brlapi support: NO Documentation: NO PIE: YES vde support: NO netmap support: NO Linux AIO support: YES Linux io_uring support: NO ATTR/XATTR support: YES Install blobs: YES malloc trim support: YES RDMA support: NO PVRDMA support: NO fdt support: NO membarrier: NO preadv support: YES fdatasync: YES madvise: YES posix_madvise: YES posix_memalign: YES libcap-ng support: NO vhost-net support: YES vhost-crypto support: YES vhost-scsi support: YES vhost-vsock support: YES vhost-user support: YES vhost-user-fs support: YES vhost-vdpa support: YES Trace backends: log spice support: NO rbd support: NO xfsctl support: NO smartcard support: NO U2F support: NO libusb: NO usb net redir: NO OpenGL support: NO OpenGL dmabufs: NO libiscsi support: NO libnfs support: NO build guest agent: YES seccomp support: NO coroutine backend: ucontext coroutine pool: YES debug stack usage: NO mutex debugging: NO crypto afalg: NO GlusterFS support: NO gcov: NO TPM support: YES libssh support: NO QOM debugging: YES Live block migration: YES lzo support: NO snappy support: NO bzip2 support: NO lzfse support: NO zstd support: NO NUMA host support: NO libxml2: NO tcmalloc support: NO jemalloc support: NO avx2 optimization: YES avx512f optimization: NO replication support: YES bochs support: YES cloop support: YES dmg support: YES qcow v1 support: YES vdi support: YES vvfat support: YES qed support: YES parallels support: YES sheepdog support: YES capstone: YES libpmem support: NO libdaxctl support: NO libudev: NO default devices: YES plugin support: NO fuzzing support: NO gdb: /usr/bin/gdb thread sanitizer: NO rng-none: NO Linux keyring: YES Found ninja-1.8.2 at /usr/bin/ninja + make -j120 /usr/bin/python3 -B /dev/shm/cfontana/qemu/meson/meson.py introspect --tests --benchmarks | /usr/bin/python3 -B scripts/mtest2make.py > Makefile.mtest ./ninjatool -t ninja2make --omit clean dist uninstall cscope TAGS ctags < build.ninja > Makefile.ninja make[1]: Entering directory '/dev/shm/cfontana/qemu/slirp' [...] CC pvh_main.o BUILD multiboot.img BUILD linuxboot.img BUILD linuxboot_dma.img BUILD kvmvapic.img BUILD multiboot.raw BUILD linuxboot.raw BUILD kvmvapic.raw BUILD linuxboot_dma.raw SIGN multiboot.bin SIGN kvmvapic.bin SIGN linuxboot.bin SIGN linuxboot_dma.bin Linking target contrib/ivshmem-client/ivshmem-client BUILD pvh.img BUILD pvh.raw In file included from tests/qapi-builtin-visit.c:14:0: /dev/shm/cfontana/qemu/include/qapi/error.h:275:10: fatal error: qapi/qapi-types-error.h: No such file or directory #include "qapi/qapi-types-error.h" ^~~~~~~~~~~~~~~~~~~~~~~~~ compilation terminated. In file included from tests/include/../test-qapi-events-sub-sub-module.h:17:0, from tests/include/test-qapi-events-sub-module.h:16, from tests/test-qapi-events.h:16, from tests/test-qapi-events.c:15: tests/include/../test-qapi-types-sub-sub-module.h:16:10: fatal error: qapi/qapi-builtin-types.h: No such file or directory #include "qapi/qapi-builtin-types.h" ^~~~~~~~~~~~~~~~~~~~~~~~~~~ compilation terminated. In file included from /dev/shm/cfontana/qemu/include/qapi/dealloc-visitor.h:17:0, from tests/qapi-builtin-types.c:14: /dev/shm/cfontana/qemu/include/qapi/visitor.h:18:10: fatal error: qapi/qapi-builtin-types.h: No such file or directory #include "qapi/qapi-builtin-types.h" ^~~~~~~~~~~~~~~~~~~~~~~~~~~ compilation terminated. In file included from /dev/shm/cfontana/qemu/include/qapi/dealloc-visitor.h:17:0, from tests/test-qapi-types.c:14: /dev/shm/cfontana/qemu/include/qapi/visitor.h:18:10: fatal error: qapi/qapi-builtin-types.h: No such file or directory #include "qapi/qapi-builtin-types.h" ^~~~~~~~~~~~~~~~~~~~~~~~~~~ compilation terminated. SIGN pvh.bin In file included from /dev/shm/cfontana/qemu/include/qapi/dealloc-visitor.h:17:0, from tests/include/test-qapi-types-sub-module.c:14: /dev/shm/cfontana/qemu/include/qapi/visitor.h:18:10: fatal error: qapi/qapi-builtin-types.h: No such file or directory #include "qapi/qapi-builtin-types.h" ^~~~~~~~~~~~~~~~~~~~~~~~~~~ compilation terminated. In file included from tests/include/test-qapi-visit-sub-module.c:14:0: /dev/shm/cfontana/qemu/include/qapi/error.h:275:10: fatal error: qapi/qapi-types-error.h: No such file or directory #include "qapi/qapi-types-error.h" ^~~~~~~~~~~~~~~~~~~~~~~~~ compilation terminated. make: *** [Makefile.ninja:1760: tests/libtestqapi.a.p/meson-generated_.._qapi-builtin-types.c.o] Error 1 make: *** Waiting for unfinished jobs.... make: *** [Makefile.ninja:1761: tests/libtestqapi.a.p/meson-generated_.._qapi-builtin-visit.c.o] Error 1 make: *** [Makefile.ninja:1766: tests/libtestqapi.a.p/meson-generated_.._test-qapi-events.c.o] Error 1 make: *** [Makefile.ninja:1770: tests/libtestqapi.a.p/meson-generated_.._test-qapi-types.c.o] Error 1 make: *** [Makefile.ninja:1775: tests/libtestqapi.a.p/meson-generated_.._include_test-qapi-types-sub-module.c.o] Error 1 make: *** [Makefile.ninja:1776: tests/libtestqapi.a.p/meson-generated_.._include_test-qapi-visit-sub-module.c.o] Error 1 In file included from tests/test-qapi-visit-sub-sub-module.c:14:0: /dev/shm/cfontana/qemu/include/qapi/error.h:275:10: fatal error: qapi/qapi-types-error.h: No such file or directory #include "qapi/qapi-types-error.h" ^~~~~~~~~~~~~~~~~~~~~~~~~ compilation terminated. In file included from tests/include/test-qapi-commands-sub-module.c:14:0: /dev/shm/cfontana/qemu/include/qapi/visitor.h:18:10: fatal error: qapi/qapi-builtin-types.h: No such file or directory #include "qapi/qapi-builtin-types.h" ^~~~~~~~~~~~~~~~~~~~~~~~~~~ compilation terminated. In file included from /dev/shm/cfontana/qemu/include/qapi/qmp/qlit.h:17:0, from tests/test-qapi-introspect.h:15, from tests/test-qapi-introspect.c:13: /dev/shm/cfontana/qemu/include/qapi/qmp/qobject.h:35:10: fatal error: qapi/qapi-builtin-types.h: No such file or directory #include "qapi/qapi-builtin-types.h" ^~~~~~~~~~~~~~~~~~~~~~~~~~~ compilation terminated. In file included from /dev/shm/cfontana/qemu/include/qapi/dealloc-visitor.h:17:0, from tests/test-qapi-types-sub-sub-module.c:14: /dev/shm/cfontana/qemu/include/qapi/visitor.h:18:10: fatal error: qapi/qapi-builtin-types.h: No such file or directory #include "qapi/qapi-builtin-types.h" ^~~~~~~~~~~~~~~~~~~~~~~~~~~ compilation terminated. make: *** [Makefile.ninja:1768: tests/libtestqapi.a.p/meson-generated_.._test-qapi-introspect.c.o] Error 1 make: *** [Makefile.ninja:1773: tests/libtestqapi.a.p/meson-generated_.._include_test-qapi-commands-sub-module.c.o] Error 1 make: *** [Makefile.ninja:1771: tests/libtestqapi.a.p/meson-generated_.._test-qapi-visit-sub-sub-module.c.o] Error 1 In file included from tests/test-qapi-commands.c:14:0: /dev/shm/cfontana/qemu/include/qapi/visitor.h:18:10: fatal error: qapi/qapi-builtin-types.h: No such file or directory #include "qapi/qapi-builtin-types.h" ^~~~~~~~~~~~~~~~~~~~~~~~~~~ compilation terminated. In file included from tests/include/../test-qapi-events-sub-sub-module.h:17:0, from tests/include/test-qapi-events-sub-module.h:16, from tests/include/test-qapi-events-sub-module.c:15: tests/include/../test-qapi-types-sub-sub-module.h:16:10: fatal error: qapi/qapi-builtin-types.h: No such file or directory #include "qapi/qapi-builtin-types.h" ^~~~~~~~~~~~~~~~~~~~~~~~~~~ compilation terminated. make: *** [Makefile.ninja:1769: tests/libtestqapi.a.p/meson-generated_.._test-qapi-types-sub-sub-module.c.o] Error 1 In file included from tests/test-qapi-events-sub-sub-module.h:17:0, from tests/test-qapi-events-sub-sub-module.c:15: tests/test-qapi-types-sub-sub-module.h:16:10: fatal error: qapi/qapi-builtin-types.h: No such file or directory #include "qapi/qapi-builtin-types.h" ^~~~~~~~~~~~~~~~~~~~~~~~~~~ compilation terminated. make: *** [Makefile.ninja:1774: tests/libtestqapi.a.p/meson-generated_.._include_test-qapi-events-sub-module.c.o] Error 1 make: *** [Makefile.ninja:1763: tests/libtestqapi.a.p/meson-generated_.._test-qapi-commands.c.o] Error 1 make: *** [Makefile.ninja:1765: tests/libtestqapi.a.p/meson-generated_.._test-qapi-events-sub-sub-module.c.o] Error 1 In file included from tests/test-qapi-visit.c:14:0: /dev/shm/cfontana/qemu/include/qapi/error.h:275:10: fatal error: qapi/qapi-types-error.h: No such file or directory #include "qapi/qapi-types-error.h" ^~~~~~~~~~~~~~~~~~~~~~~~~ compilation terminated. In file included from tests/test-qapi-commands-sub-sub-module.c:14:0: /dev/shm/cfontana/qemu/include/qapi/visitor.h:18:10: fatal error: qapi/qapi-builtin-types.h: No such file or directory #include "qapi/qapi-builtin-types.h" ^~~~~~~~~~~~~~~~~~~~~~~~~~~ compilation terminated. In file included from tests/include/../test-qapi-events-sub-sub-module.h:17:0, from tests/include/test-qapi-events-sub-module.h:16, from tests/include/test-qapi-events-sub-module.c:15: tests/include/../test-qapi-types-sub-sub-module.h:16:10: fatal error: qapi/qapi-builtin-types.h: No such file or directory #include "qapi/qapi-builtin-types.h" ^~~~~~~~~~~~~~~~~~~~~~~~~~~ compilation terminated. make: *** [Makefile.ninja:1769: tests/libtestqapi.a.p/meson-generated_.._test-qapi-types-sub-sub-module.c.o] Error 1 In file included from tests/test-qapi-events-sub-sub-module.h:17:0, from tests/test-qapi-events-sub-sub-module.c:15: tests/test-qapi-types-sub-sub-module.h:16:10: fatal error: qapi/qapi-builtin-types.h: No such file or directory #include "qapi/qapi-builtin-types.h" ^~~~~~~~~~~~~~~~~~~~~~~~~~~ compilation terminated. make: *** [Makefile.ninja:1774: tests/libtestqapi.a.p/meson-generated_.._include_test-qapi-events-sub-module.c.o] Error 1 make: *** [Makefile.ninja:1763: tests/libtestqapi.a.p/meson-generated_.._test-qapi-commands.c.o] Error 1 make: *** [Makefile.ninja:1765: tests/libtestqapi.a.p/meson-generated_.._test-qapi-events-sub-sub-module.c.o] Error 1 In file included from tests/test-qapi-visit.c:14:0: /dev/shm/cfontana/qemu/include/qapi/error.h:275:10: fatal error: qapi/qapi-types-error.h: No such file or directory #include "qapi/qapi-types-error.h" ^~~~~~~~~~~~~~~~~~~~~~~~~ compilation terminated. In file included from tests/test-qapi-commands-sub-sub-module.c:14:0: /dev/shm/cfontana/qemu/include/qapi/visitor.h:18:10: fatal error: qapi/qapi-builtin-types.h: No such file or directory #include "qapi/qapi-builtin-types.h" ^~~~~~~~~~~~~~~~~~~~~~~~~~~ compilation terminated. In file included from tests/include/../test-qapi-events-sub-sub-module.h:17:0, from tests/include/test-qapi-events-sub-module.h:16, from tests/include/test-qapi-events-sub-module.c:15: tests/include/../test-qapi-types-sub-sub-module.h:16:10: fatal error: qapi/qapi-builtin-types.h: No such file or directory #include "qapi/qapi-builtin-types.h" ^~~~~~~~~~~~~~~~~~~~~~~~~~~ compilation terminated. make: *** [Makefile.ninja:1769: tests/libtestqapi.a.p/meson-generated_.._test-qapi-types-sub-sub-module.c.o] Error 1 In file included from tests/test-qapi-events-sub-sub-module.h:17:0, from tests/test-qapi-events-sub-sub-module.c:15: tests/test-qapi-types-sub-sub-module.h:16:10: fatal error: qapi/qapi-builtin-types.h: No such file or directory #include "qapi/qapi-builtin-types.h" ^~~~~~~~~~~~~~~~~~~~~~~~~~~ compilation terminated. make: *** [Makefile.ninja:1774: tests/libtestqapi.a.p/meson-generated_.._include_test-qapi-events-sub-module.c.o] Error 1 make: *** [Makefile.ninja:1763: tests/libtestqapi.a.p/meson-generated_.._test-qapi-commands.c.o] Error 1 make: *** [Makefile.ninja:1765: tests/libtestqapi.a.p/meson-generated_.._test-qapi-events-sub-sub-module.c.o] Error 1 In file included from tests/test-qapi-visit.c:14:0: /dev/shm/cfontana/qemu/include/qapi/error.h:275:10: fatal error: qapi/qapi-types-error.h: No such file or directory #include "qapi/qapi-types-error.h" ^~~~~~~~~~~~~~~~~~~~~~~~~ compilation terminated. In file included from tests/test-qapi-commands-sub-sub-module.c:14:0: /dev/shm/cfontana/qemu/include/qapi/visitor.h:18:10: fatal error: qapi/qapi-builtin-types.h: No such file or directory #include "qapi/qapi-builtin-types.h" ^~~~~~~~~~~~~~~~~~~~~~~~~~~ compilation terminated. In file included from tests/include/../test-qapi-commands-sub-sub-module.h:16:0, from tests/include/test-qapi-commands-sub-module.h:16, from tests/test-qapi-commands.h:16, from tests/test-qapi-init-commands.c:14: tests/include/../test-qapi-types-sub-sub-module.h:16:10: fatal error: qapi/qapi-builtin-types.h: No such file or directory #include "qapi/qapi-builtin-types.h" ^~~~~~~~~~~~~~~~~~~~~~~~~~~ compilation terminated. make: *** [Makefile.ninja:1772: tests/libtestqapi.a.p/meson-generated_.._test-qapi-visit.c.o] Error 1 make: *** [Makefile.ninja:1762: tests/libtestqapi.a.p/meson-generated_.._test-qapi-commands-sub-sub-module.c.o] Error 1 make: *** [Makefile.ninja:1767: tests/libtestqapi.a.p/meson-generated_.._test-qapi-init-commands.c.o] Error 1 warning: execution failed: build.sh You can fix the problem, and then run git rebase --continue ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Meson can't recover from deletion of generated QAPI file(s) 2020-09-09 15:05 ` Claudio Fontana @ 2020-09-09 15:10 ` Paolo Bonzini 2020-09-09 21:25 ` Claudio Fontana 0 siblings, 1 reply; 11+ messages in thread From: Paolo Bonzini @ 2020-09-09 15:10 UTC (permalink / raw) To: Claudio Fontana, Marc-André Lureau Cc: Philippe Mathieu-Daudé, Markus Armbruster, qemu-devel On 09/09/20 17:05, Claudio Fontana wrote: > Hmm, I still encounter problems: And another: diff --git a/tests/meson.build b/tests/meson.build index 998e4c48f9..95789f43b3 100644 --- a/tests/meson.build +++ b/tests/meson.build @@ -56,7 +56,7 @@ test_qapi_files = custom_target('Test QAPI files', # perhaps change qapi_gen to replace / with _, like Meson itself does? subdir('include') -libtestqapi = static_library('testqapi', sources: [test_qapi_files, test_qapi_outputs_extra]) +libtestqapi = static_library('testqapi', sources: [test_qapi_files, genh, test_qapi_outputs_extra]) testqapi = declare_dependency(link_with: libtestqapi) testblock = declare_dependency(dependencies: [block], sources: 'iothread.c') Paolo ^ permalink raw reply related [flat|nested] 11+ messages in thread
* Re: Meson can't recover from deletion of generated QAPI file(s) 2020-09-09 15:10 ` Paolo Bonzini @ 2020-09-09 21:25 ` Claudio Fontana 0 siblings, 0 replies; 11+ messages in thread From: Claudio Fontana @ 2020-09-09 21:25 UTC (permalink / raw) To: Paolo Bonzini, Marc-André Lureau Cc: Philippe Mathieu-Daudé, Markus Armbruster, qemu-devel On 9/9/20 5:10 PM, Paolo Bonzini wrote: > On 09/09/20 17:05, Claudio Fontana wrote: >> Hmm, I still encounter problems: > > And another: > > diff --git a/tests/meson.build b/tests/meson.build > index 998e4c48f9..95789f43b3 100644 > --- a/tests/meson.build > +++ b/tests/meson.build > @@ -56,7 +56,7 @@ test_qapi_files = custom_target('Test QAPI files', > # perhaps change qapi_gen to replace / with _, like Meson itself does? > subdir('include') > > -libtestqapi = static_library('testqapi', sources: [test_qapi_files, test_qapi_outputs_extra]) > +libtestqapi = static_library('testqapi', sources: [test_qapi_files, genh, test_qapi_outputs_extra]) > testqapi = declare_dependency(link_with: libtestqapi) > > testblock = declare_dependency(dependencies: [block], sources: 'iothread.c') > > Paolo > adding this one as well fixes it for me, thanks! Tested-by: Claudio Fontana <cfontana@suse.de> ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Meson can't recover from deletion of generated QAPI file(s) 2020-09-09 12:37 Meson can't recover from deletion of generated QAPI file(s) Markus Armbruster 2020-09-09 13:06 ` Philippe Mathieu-Daudé @ 2020-09-09 14:00 ` Marc-André Lureau 2020-09-09 14:34 ` Paolo Bonzini 2 siblings, 0 replies; 11+ messages in thread From: Marc-André Lureau @ 2020-09-09 14:00 UTC (permalink / raw) To: Markus Armbruster; +Cc: Paolo Bonzini, QEMU [-- Attachment #1: Type: text/plain, Size: 1165 bytes --] Hi On Wed, Sep 9, 2020 at 4:37 PM Markus Armbruster <armbru@redhat.com> wrote: > Watch this: > > $ rm qapi/qapi-types-error.h > $ make > Generating qemu-version.h with a meson_exe.py custom command > Compiling C object qom/libqom.fa.p/object_interfaces.c.o > In file included from ../qom/object_interfaces.c:4: > /work/armbru/qemu/include/qapi/error.h:275:10: fatal error: > qapi/qapi-types-error.h: No such file or directory > 275 | #include "qapi/qapi-types-error.h" > | ^~~~~~~~~~~~~~~~~~~~~~~~~ > compilation terminated. > make: *** [Makefile.ninja:348: qom/libqom.fa.p/object_interfaces.c.o] > Error 1 > > To recover, I have to run qapi-gen.py manually and exactly right, or > blow away the build tree and start over. > > The old build system did what a build system should: it remade the files > that are missing or out of date. > > I'm still too clueless about Meson to debug this, but I hope I can learn > from watching you fix it. > It looks like the ninja build handles that fine, so this is most likely a ninja2make conversion issue. -- Marc-André Lureau [-- Attachment #2: Type: text/html, Size: 1633 bytes --] ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Meson can't recover from deletion of generated QAPI file(s) 2020-09-09 12:37 Meson can't recover from deletion of generated QAPI file(s) Markus Armbruster 2020-09-09 13:06 ` Philippe Mathieu-Daudé 2020-09-09 14:00 ` Marc-André Lureau @ 2020-09-09 14:34 ` Paolo Bonzini 2020-09-10 7:44 ` Markus Armbruster 2 siblings, 1 reply; 11+ messages in thread From: Paolo Bonzini @ 2020-09-09 14:34 UTC (permalink / raw) To: Markus Armbruster, Marc-André Lureau Cc: Philippe Mathieu-Daudé, qemu-devel On 09/09/20 14:37, Markus Armbruster wrote: > Watch this: > > $ rm qapi/qapi-types-error.h > $ make > Generating qemu-version.h with a meson_exe.py custom command > Compiling C object qom/libqom.fa.p/object_interfaces.c.o > In file included from ../qom/object_interfaces.c:4: > /work/armbru/qemu/include/qapi/error.h:275:10: fatal error: qapi/qapi-types-error.h: No such file or directory > 275 | #include "qapi/qapi-types-error.h" > | ^~~~~~~~~~~~~~~~~~~~~~~~~ > compilation terminated. > make: *** [Makefile.ninja:348: qom/libqom.fa.p/object_interfaces.c.o] Error 1 > > To recover, I have to run qapi-gen.py manually and exactly right, or > blow away the build tree and start over. > > The old build system did what a build system should: it remade the files > that are missing or out of date. > > I'm still too clueless about Meson to debug this, but I hope I can learn > from watching you fix it. Unfortunately you won't learn much about Meson, you would learn that Make is messy but you don't need that. You can also learn a little bit about the new design of the QEMU build system though, so I'll explain and not just send a patch. The bad news this tells you about the build system is that, when debugging an issue, you have to figure out if it's a bug in Meson, in the meson.build files, or in ninja2make. Of course the second is the common case, but you never know especially now that there are more people using ninja2make in anger. Generating Makefile.ninja gets finicky because ninja (while it has other things I don't like) is a little more expressive than Make as far as simple build rules are concerned, therefore it doesn't need the stamp file trick. So while we there may be one or two more bugs like this one down the road, ninja2make should not be an issue as soon as its teething problems are solved. (As an aside: the GNU Make 4.3 "grouped targets" feature can sometimes eliminate stamp files, but it would not help here. The stamp file has another feature, namely the custom command can decide not to touch its outputs if they won't change. This avoid more rebuilds. Grouped targets don't have a way to do with that). The good news is that there's an easy(ish) way to do this. The build system is quite "linear" in how it works, so the first step should be to look at build.ninja and see what the rules are like. Here you'd see something like: build long list of files: CUSTOM_COMMAND actual prerequisites COMMAND = ... description = Generating$ shared$ QAPI$ source$ files Your twenty-plus-years-of-writing-Makefiles spidey sense will tingle, as you can figure out that this is not going to be trivial to convert to Makefiles. If you open Makefile.ninja you see the familiar stamp file trick: long list of files: CUSTOM_COMMAND@57579de3eef.stamp; @: CUSTOM_COMMAND@57579de3eef.stamp: actual prerequisites $(ninja-command-restat) and that's where the bug is. If you delete one of the output files, Make only runs ":" and does not rebuild it. One solution is to add: ifneq (long list of files, $(wildcard long list of files)) .PHONY: CUSTOM_COMMAND@57579de3eef.stamp endif This way, if any of the prerequites is missing (not just older than the stamp file), the rule for CUSTOM_COMMAND@57579de3eef.stamp will always be executed. This is fairly simple to do: diff --git a/scripts/ninjatool.py b/scripts/ninjatool.py index 627a1cab45..6f0e35c727 100755 --- a/scripts/ninjatool.py +++ b/scripts/ninjatool.py @@ -908,6 +908,9 @@ class Ninja2Make(NinjaParserEventsWithVars): else: stamp = '%s@%s.stamp' % (rule, sha1_text(targets)[0:11]) self.print('%s: %s; @:' % (targets, stamp)) + self.print('ifneq (%s, $(wildcard %s))' % (targets, targets)) + self.print('.PHONY: %s' % (stamp, )) + self.print('endif') self.print('%s: %s | %s; ${ninja-command-restat}' % (stamp, inputs, orderonly)) self.rule_targets[rule].append(stamp) self.stamp_targets[rule].append(stamp) To avoid this whole class of issues we could just use ninja to build QEMU (Make would launch it, so there would still be no user-facing changes). ninja2make's main strength was that it supported incremental conversion, but right now all of the binaries are built by Meson therefore it's not really *necessary* anymore. Dropping ninja2make removes a relatively expensive part of the build as well as a nontrivial amount of code. Another advantage would be that ninja tracks command lines and automatically rebuilds things if the command line has changed. This is quite hard and expensive to do with Make so ninja2make does not even try, but it has bitten Philippe. Of course, the main disadvantage is that it adds another dependency. I will send the above patch formally soonish, but I wouldn't mind if somebody else helped merging it. Paolo ^ permalink raw reply related [flat|nested] 11+ messages in thread
* Re: Meson can't recover from deletion of generated QAPI file(s) 2020-09-09 14:34 ` Paolo Bonzini @ 2020-09-10 7:44 ` Markus Armbruster 0 siblings, 0 replies; 11+ messages in thread From: Markus Armbruster @ 2020-09-10 7:44 UTC (permalink / raw) To: Paolo Bonzini Cc: Marc-André Lureau, Philippe Mathieu-Daudé, qemu-devel Paolo Bonzini <pbonzini@redhat.com> writes: > On 09/09/20 14:37, Markus Armbruster wrote: >> Watch this: >> >> $ rm qapi/qapi-types-error.h >> $ make >> Generating qemu-version.h with a meson_exe.py custom command >> Compiling C object qom/libqom.fa.p/object_interfaces.c.o >> In file included from ../qom/object_interfaces.c:4: >> /work/armbru/qemu/include/qapi/error.h:275:10: fatal error: qapi/qapi-types-error.h: No such file or directory >> 275 | #include "qapi/qapi-types-error.h" >> | ^~~~~~~~~~~~~~~~~~~~~~~~~ >> compilation terminated. >> make: *** [Makefile.ninja:348: qom/libqom.fa.p/object_interfaces.c.o] Error 1 >> >> To recover, I have to run qapi-gen.py manually and exactly right, or >> blow away the build tree and start over. >> >> The old build system did what a build system should: it remade the files >> that are missing or out of date. >> >> I'm still too clueless about Meson to debug this, but I hope I can learn >> from watching you fix it. > > Unfortunately you won't learn much about Meson, you would learn that > Make is messy but you don't need that. You can also learn a little bit > about the new design of the QEMU build system though, so I'll explain > and not just send a patch. Appreciated! > The bad news this tells you about the build system is that, when > debugging an issue, you have to figure out if it's a bug in Meson, > in the meson.build files, or in ninja2make. Of course the second > is the common case, but you never know especially now that there are > more people using ninja2make in anger. > > Generating Makefile.ninja gets finicky because ninja (while it has other > things I don't like) is a little more expressive than Make as far as > simple build rules are concerned, therefore it doesn't need the stamp > file trick. So while we there may be one or two more bugs like this one > down the road, ninja2make should not be an issue as soon as its teething > problems are solved. A Make replacement must address its issues to be credible. Make's traditional inability to directly express "rule updates multiple files" and "rule may or may not actually change its target(s)" has always been an issue. Countless Make users have had to learn the stamp file work-around, and how to cope with its drawbacks. I distinctly remember grappling with it back in the 90s. > (As an aside: the GNU Make 4.3 "grouped targets" feature can sometimes > eliminate stamp files, but it would not help here. The stamp file has > another feature, namely the custom command can decide not to touch > its outputs if they won't change. This avoid more rebuilds. This is *essential* when generating headers. Without it, touching any QAPI module or any part of the QAPI generator recompiles pretty much everything. > Grouped > targets don't have a way to do with that). Yes. Also, TIL grouped targets :) > The good news is that there's an easy(ish) way to do this. The build > system is quite "linear" in how it works, so the first step should be to > look at build.ninja and see what the rules are like. Here you'd see > something like: > > build long list of files: CUSTOM_COMMAND actual prerequisites > COMMAND = ... > description = Generating$ shared$ QAPI$ source$ files > > Your twenty-plus-years-of-writing-Makefiles spidey sense will tingle, as > you can figure out that this is not going to be trivial to convert to > Makefiles. If you open Makefile.ninja you see the familiar stamp file > trick: > > long list of files: CUSTOM_COMMAND@57579de3eef.stamp; @: > CUSTOM_COMMAND@57579de3eef.stamp: actual prerequisites > $(ninja-command-restat) > > and that's where the bug is. If you delete one of the output files, Make > only runs ":" and does not rebuild it. One solution is to add: > > ifneq (long list of files, $(wildcard long list of files)) > .PHONY: CUSTOM_COMMAND@57579de3eef.stamp > endif > > This way, if any of the prerequites is missing (not just older than the > stamp file), the rule for CUSTOM_COMMAND@57579de3eef.stamp will always > be executed. Neat trick! Now I remember the old Makefiles actually didn't recover all by themselves either, because they didn't use this trick. Manual recovery was "easy", though: remove the stamp file. I ran into the "can't recover from deletion of generated QAPI file(s)" issue when I updated a silly script of mine that helps me diff them. To force regeneration, the script removes them (relying on ccache to keep compilation time in check). Pre-Meson, it removed the stamp files. I didn't see any in meson.build, so I dropped their removal, nothing worked, and I got quite confused. > This is fairly simple to do: > > diff --git a/scripts/ninjatool.py b/scripts/ninjatool.py > index 627a1cab45..6f0e35c727 100755 > --- a/scripts/ninjatool.py > +++ b/scripts/ninjatool.py > @@ -908,6 +908,9 @@ class Ninja2Make(NinjaParserEventsWithVars): > else: > stamp = '%s@%s.stamp' % (rule, sha1_text(targets)[0:11]) > self.print('%s: %s; @:' % (targets, stamp)) > + self.print('ifneq (%s, $(wildcard %s))' % (targets, targets)) > + self.print('.PHONY: %s' % (stamp, )) > + self.print('endif') > self.print('%s: %s | %s; ${ninja-command-restat}' % (stamp, inputs, orderonly)) > self.rule_targets[rule].append(stamp) > self.stamp_targets[rule].append(stamp) > > To avoid this whole class of issues we could just use ninja to build QEMU > (Make would launch it, so there would still be no user-facing changes). > ninja2make's main strength was that it supported incremental conversion, > but right now all of the binaries are built by Meson therefore it's not > really *necessary* anymore. Dropping ninja2make removes a relatively > expensive part of the build as well as a nontrivial amount of code. Makes sense to me. > Another advantage would be that ninja tracks command lines and automatically > rebuilds things if the command line has changed. This is quite hard and > expensive to do with Make so ninja2make does not even try, but it has bitten > Philippe. > > Of course, the main disadvantage is that it adds another dependency. Yes. Drop in the bucket? We already depend on ninja-the-language, we just feed it to ninja2make instead of ninja. > I will send the above patch formally soonish, but I wouldn't mind if somebody > else helped merging it. Thank you very much for taking the time to explain! ^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2020-09-10 10:14 UTC | newest] Thread overview: 11+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2020-09-09 12:37 Meson can't recover from deletion of generated QAPI file(s) Markus Armbruster 2020-09-09 13:06 ` Philippe Mathieu-Daudé 2020-09-09 13:35 ` Claudio Fontana 2020-09-09 14:35 ` Paolo Bonzini 2020-09-09 14:36 ` Claudio Fontana 2020-09-09 15:05 ` Claudio Fontana 2020-09-09 15:10 ` Paolo Bonzini 2020-09-09 21:25 ` Claudio Fontana 2020-09-09 14:00 ` Marc-André Lureau 2020-09-09 14:34 ` Paolo Bonzini 2020-09-10 7:44 ` Markus Armbruster
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).