[PULL 0/8] s390x and misc fixes

[PULL 0/8] s390x and misc fixes

[Thomas Huth]

 Hi Peter!

The following changes since commit 352998df1c53b366413690d95b35f76d0721ebed:

  Merge tag 'i2c-20220314' of https://github.com/philmd/qemu into staging (2022-03-14 14:39:33 +0000)

are available in the Git repository at:

  https://gitlab.com/thuth/qemu.git tags/pull-request-2022-03-15

for you to fetch changes up to 36149534792dcf07a3c59867f967eaee23ab906c:

  meson: Update to version 0.61.3 (2022-03-15 10:32:36 +0100)

----------------------------------------------------------------
* Fixes for s390x branch instruction emulation
* Fixes for the tests/avocado/boot_linux.py:BootLinuxS390X test
* Fix for "-cpu help" output
* Bump meson to 0.61.3 to fix stderr log of the iotests

----------------------------------------------------------------
Alex Bennée (1):
      tests/tcg: drop -cpu max from s390x sha512-mvx invocation

Beraldo Leal (1):
      tests/avocado: start PhoneServer upfront

Ilya Leoshkevich (3):
      s390x/tcg: Fix BRASL with a large negative offset
      s390x/tcg: Fix BRCL with a large negative offset
      tests/tcg/s390x: Test BRASL and BRCL with large negative offsets

Philippe Mathieu-Daudé (1):
      softmmu: List CPU types again

Richard Henderson (1):
      accel/tcg: Fix cpu_ldq_be_mmu typo

Thomas Huth (1):
      meson: Update to version 0.61.3

 accel/tcg/cputlb.c                     |  2 +-
 configure                              |  2 +-
 cpu.c                                  |  9 +++++
 include/exec/cpu-common.h              |  2 +
 include/sysemu/cpus.h                  |  2 -
 meson                                  |  2 +-
 softmmu/cpus.c                         |  8 ----
 target/s390x/tcg/translate.c           |  4 +-
 tests/avocado/avocado_qemu/__init__.py | 13 ++++---
 tests/tcg/s390x/Makefile.target        |  3 +-
 tests/tcg/s390x/branch-relative-long.c | 68 ++++++++++++++++++++++++++++++++++
 11 files changed, 92 insertions(+), 23 deletions(-)
 create mode 100644 tests/tcg/s390x/branch-relative-long.c

[PULL 1/8] tests/tcg: drop -cpu max from s390x sha512-mvx invocation

[Thomas Huth]

From: Alex Bennée <alex.bennee@linaro.org>

With -cpu max we get a warning:

  qemu-s390x: warning: 'msa5-base' requires 'kimd-sha-512'.

But dropping the -cpu max and it still runs fine.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20220309112248.4083619-1-alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: David Hildenbrand <david@redhat.com>
Tested-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 tests/tcg/s390x/Makefile.target | 2 --
 1 file changed, 2 deletions(-)

diff --git a/tests/tcg/s390x/Makefile.target b/tests/tcg/s390x/Makefile.target
index 257c568c58..7aa502a557 100644
--- a/tests/tcg/s390x/Makefile.target
+++ b/tests/tcg/s390x/Makefile.target
@@ -34,6 +34,4 @@ sha512-mvx: CFLAGS=-march=z13 -mvx -O3
 sha512-mvx: sha512.c
 	$(CC) $(CFLAGS) $(EXTRA_CFLAGS) $< -o $@ $(LDFLAGS)
 
-run-sha512-mvx: QEMU_OPTS+=-cpu max
-
 TESTS+=sha512-mvx
-- 
2.27.0

[PULL 2/8] s390x/tcg: Fix BRASL with a large negative offset

[Thomas Huth]

From: Ilya Leoshkevich <iii@linux.ibm.com>

When RI2 is 0x80000000, qemu enters an infinite loop instead of jumping
backwards. Fix by adding a missing cast, like in in2_ri2().

Fixes: 8ac33cdb8bfb ("Convert BRANCH AND SAVE")
Signed-off-by: Ilya Leoshkevich <iii@linux.ibm.com>
Message-Id: <20220314104232.675863-2-iii@linux.ibm.com>
Reviewed-by: David Hildenbrand <david@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 target/s390x/tcg/translate.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/target/s390x/tcg/translate.c b/target/s390x/tcg/translate.c
index 904b51542f..41c8696185 100644
--- a/target/s390x/tcg/translate.c
+++ b/target/s390x/tcg/translate.c
@@ -1597,7 +1597,7 @@ static DisasJumpType op_bal(DisasContext *s, DisasOps *o)
 static DisasJumpType op_basi(DisasContext *s, DisasOps *o)
 {
     pc_to_link_info(o->out, s, s->pc_tmp);
-    return help_goto_direct(s, s->base.pc_next + 2 * get_field(s, i2));
+    return help_goto_direct(s, s->base.pc_next + (int64_t)get_field(s, i2) * 2);
 }
 
 static DisasJumpType op_bc(DisasContext *s, DisasOps *o)
-- 
2.27.0

[PULL 3/8] s390x/tcg: Fix BRCL with a large negative offset

[Thomas Huth]

From: Ilya Leoshkevich <iii@linux.ibm.com>

When RI2 is 0x80000000, qemu enters an infinite loop instead of jumping
backwards. Fix by adding a missing cast, like in in2_ri2().

Fixes: 7233f2ed1717 ("target-s390: Convert BRANCH ON CONDITION")
Signed-off-by: Ilya Leoshkevich <iii@linux.ibm.com>
Message-Id: <20220314104232.675863-3-iii@linux.ibm.com>
Reviewed-by: David Hildenbrand <david@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 target/s390x/tcg/translate.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/target/s390x/tcg/translate.c b/target/s390x/tcg/translate.c
index 41c8696185..5acfc0ff9b 100644
--- a/target/s390x/tcg/translate.c
+++ b/target/s390x/tcg/translate.c
@@ -1201,7 +1201,7 @@ static DisasJumpType help_branch(DisasContext *s, DisasCompare *c,
                                  bool is_imm, int imm, TCGv_i64 cdest)
 {
     DisasJumpType ret;
-    uint64_t dest = s->base.pc_next + 2 * imm;
+    uint64_t dest = s->base.pc_next + (int64_t)imm * 2;
     TCGLabel *lab;
 
     /* Take care of the special cases first.  */
-- 
2.27.0

[PULL 4/8] tests/tcg/s390x: Test BRASL and BRCL with large negative offsets

[Thomas Huth]

From: Ilya Leoshkevich <iii@linux.ibm.com>

Add a small test in order to prevent regressions.

Signed-off-by: Ilya Leoshkevich <iii@linux.ibm.com>
Message-Id: <20220314104232.675863-4-iii@linux.ibm.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: David Hildenbrand <david@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 tests/tcg/s390x/Makefile.target        |  1 +
 tests/tcg/s390x/branch-relative-long.c | 68 ++++++++++++++++++++++++++
 2 files changed, 69 insertions(+)
 create mode 100644 tests/tcg/s390x/branch-relative-long.c

diff --git a/tests/tcg/s390x/Makefile.target b/tests/tcg/s390x/Makefile.target
index 7aa502a557..f0d474a245 100644
--- a/tests/tcg/s390x/Makefile.target
+++ b/tests/tcg/s390x/Makefile.target
@@ -15,6 +15,7 @@ TESTS+=mvc
 TESTS+=shift
 TESTS+=trap
 TESTS+=signals-s390x
+TESTS+=branch-relative-long
 
 ifneq ($(HAVE_GDB_BIN),)
 GDB_SCRIPT=$(SRC_PATH)/tests/guest-debug/run-test.py
diff --git a/tests/tcg/s390x/branch-relative-long.c b/tests/tcg/s390x/branch-relative-long.c
new file mode 100644
index 0000000000..94219afcad
--- /dev/null
+++ b/tests/tcg/s390x/branch-relative-long.c
@@ -0,0 +1,68 @@
+#include <stddef.h>
+#include <stdio.h>
+#include <string.h>
+#include <sys/mman.h>
+
+#define DEFINE_ASM(_name, _code) \
+    extern const char _name[]; \
+    extern const char _name ## _end[]; \
+    asm("    .globl " #_name "\n" \
+        #_name ":\n" \
+        "    " _code "\n" \
+        "    .globl " #_name "_end\n" \
+        #_name "_end:\n");
+
+DEFINE_ASM(br_r14, "br %r14");
+DEFINE_ASM(brasl_r0, "brasl %r0,.-0x100000000");
+DEFINE_ASM(brcl_0xf, "brcl 0xf,.-0x100000000");
+
+struct test {
+    const char *code;
+    const char *code_end;
+};
+
+static const struct test tests[] = {
+    {
+        .code = brasl_r0,
+        .code_end = brasl_r0_end,
+    },
+    {
+        .code = brcl_0xf,
+        .code_end = brcl_0xf_end,
+    },
+};
+
+int main(void)
+{
+    unsigned char *buf;
+    size_t length = 0;
+    size_t i;
+
+    for (i = 0; i < sizeof(tests) / sizeof(tests[0]); i++) {
+        size_t test_length = 0x100000000 + (tests[i].code_end - tests[i].code);
+
+        if (test_length > length) {
+            length = test_length;
+        }
+    }
+
+    buf = mmap(NULL, length, PROT_READ | PROT_WRITE | PROT_EXEC,
+               MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
+    if (buf == MAP_FAILED) {
+        perror("SKIP: mmap() failed");
+        return 0;
+    }
+
+    memcpy(buf, br_r14, br_r14_end - br_r14);
+    for (i = 0; i < sizeof(tests) / sizeof(tests[0]); i++) {
+        void (*code)(void) = (void *)(buf + 0x100000000);
+
+        memcpy(code, tests[i].code, tests[i].code_end - tests[i].code);
+        code();
+        memset(code, 0, tests[i].code_end - tests[i].code);
+    }
+
+    munmap(buf, length);
+
+    return 0;
+}
-- 
2.27.0

Re: [PULL 4/8] tests/tcg/s390x: Test BRASL and BRCL with large negative offsets

[Thomas Huth]

On 15/03/2022 12.20, Thomas Huth wrote:
> From: Ilya Leoshkevich <iii@linux.ibm.com>
> 
> Add a small test in order to prevent regressions.
> 
> Signed-off-by: Ilya Leoshkevich <iii@linux.ibm.com>
> Message-Id: <20220314104232.675863-4-iii@linux.ibm.com>
> Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
> Reviewed-by: David Hildenbrand <david@redhat.com>
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
>   tests/tcg/s390x/Makefile.target        |  1 +
>   tests/tcg/s390x/branch-relative-long.c | 68 ++++++++++++++++++++++++++
>   2 files changed, 69 insertions(+)
>   create mode 100644 tests/tcg/s390x/branch-relative-long.c
> 
> diff --git a/tests/tcg/s390x/Makefile.target b/tests/tcg/s390x/Makefile.target
> index 7aa502a557..f0d474a245 100644
> --- a/tests/tcg/s390x/Makefile.target
> +++ b/tests/tcg/s390x/Makefile.target
> @@ -15,6 +15,7 @@ TESTS+=mvc
>   TESTS+=shift
>   TESTS+=trap
>   TESTS+=signals-s390x
> +TESTS+=branch-relative-long
>   
>   ifneq ($(HAVE_GDB_BIN),)
>   GDB_SCRIPT=$(SRC_PATH)/tests/guest-debug/run-test.py
> diff --git a/tests/tcg/s390x/branch-relative-long.c b/tests/tcg/s390x/branch-relative-long.c
> new file mode 100644
> index 0000000000..94219afcad
> --- /dev/null
> +++ b/tests/tcg/s390x/branch-relative-long.c
> @@ -0,0 +1,68 @@
> +#include <stddef.h>
> +#include <stdio.h>
> +#include <string.h>
> +#include <sys/mman.h>
> +
> +#define DEFINE_ASM(_name, _code) \
> +    extern const char _name[]; \
> +    extern const char _name ## _end[]; \
> +    asm("    .globl " #_name "\n" \
> +        #_name ":\n" \
> +        "    " _code "\n" \
> +        "    .globl " #_name "_end\n" \
> +        #_name "_end:\n");
> +
> +DEFINE_ASM(br_r14, "br %r14");
> +DEFINE_ASM(brasl_r0, "brasl %r0,.-0x100000000");
> +DEFINE_ASM(brcl_0xf, "brcl 0xf,.-0x100000000");

  Hi Ilya,

I just noticed that this does not compile with Clang
(version 13.0.1):

<inline asm>:8:15: error: offset out of range
     brasl %r0,.-0x100000000
               ^
<inline asm>:13:14: error: offset out of range
     brcl 0xf,.-0x100000000
              ^
2 errors generated.

Any ideas how to fix this? Could we decrease the offset a little bit? Or 
maybe detect Clang and run with -no-integrated-as in that case?

  Thomas

[PULL 5/8] tests/avocado: start PhoneServer upfront

[Thomas Huth]

From: Beraldo Leal <bleal@redhat.com>

Race conditions can happen with the current code, because the port that
was available might not be anymore by the time the server is started.

By setting the port to 0, PhoneServer it will use the OS default
behavior to get a free port, then we save this information so we can
later configure the guest.

Suggested-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Beraldo Leal <bleal@redhat.com>
Message-Id: <20220311171127.2189534-1-bleal@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 tests/avocado/avocado_qemu/__init__.py | 13 +++++++------
 1 file changed, 7 insertions(+), 6 deletions(-)

diff --git a/tests/avocado/avocado_qemu/__init__.py b/tests/avocado/avocado_qemu/__init__.py
index 9b056b5ce5..ac85e36a4d 100644
--- a/tests/avocado/avocado_qemu/__init__.py
+++ b/tests/avocado/avocado_qemu/__init__.py
@@ -18,7 +18,7 @@
 import uuid
 
 import avocado
-from avocado.utils import cloudinit, datadrainer, network, process, ssh, vmimage
+from avocado.utils import cloudinit, datadrainer, process, ssh, vmimage
 from avocado.utils.path import find_command
 
 #: The QEMU build root directory.  It may also be the source directory
@@ -602,9 +602,6 @@ def prepare_cloudinit(self, ssh_pubkey=None):
         self.log.info('Preparing cloudinit image')
         try:
             cloudinit_iso = os.path.join(self.workdir, 'cloudinit.iso')
-            self.phone_home_port = network.find_free_port()
-            if not self.phone_home_port:
-                self.cancel('Failed to get a free port')
             pubkey_content = None
             if ssh_pubkey:
                 with open(ssh_pubkey) as pubkey:
@@ -614,7 +611,7 @@ def prepare_cloudinit(self, ssh_pubkey=None):
                           password=self.password,
                           # QEMU's hard coded usermode router address
                           phone_home_host='10.0.2.2',
-                          phone_home_port=self.phone_home_port,
+                          phone_home_port=self.phone_server.server_port,
                           authorized_key=pubkey_content)
         except Exception:
             self.cancel('Failed to prepare the cloudinit image')
@@ -625,6 +622,8 @@ def set_up_boot(self):
         self.vm.add_args('-drive', 'file=%s' % path)
 
     def set_up_cloudinit(self, ssh_pubkey=None):
+        self.phone_server = cloudinit.PhoneHomeServer(('0.0.0.0', 0),
+                                                      self.name)
         cloudinit_iso = self.prepare_cloudinit(ssh_pubkey)
         self.vm.add_args('-drive', 'file=%s,format=raw' % cloudinit_iso)
 
@@ -635,7 +634,9 @@ def launch_and_wait(self, set_up_ssh_connection=True):
                                                  logger=self.log.getChild('console'))
         console_drainer.start()
         self.log.info('VM launched, waiting for boot confirmation from guest')
-        cloudinit.wait_for_phone_home(('0.0.0.0', self.phone_home_port), self.name)
+        while not self.phone_server.instance_phoned_back:
+            self.phone_server.handle_request()
+
         if set_up_ssh_connection:
             self.log.info('Setting up the SSH connection')
             self.ssh_connect(self.username, self.ssh_key)
-- 
2.27.0

[PULL 6/8] accel/tcg: Fix cpu_ldq_be_mmu typo

[Thomas Huth]

From: Richard Henderson <richard.henderson@linaro.org>

In the conversion to cpu_ld_*_mmu, the retaddr parameter
was corrupted in the one case of cpu_ldq_be_mmu.

Fixes: f83bcecb1 ("accel/tcg: Add cpu_{ld,st}*_mmu interfaces")
Resolves: https://gitlab.com/qemu-project/qemu/-/issues/902
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220315002506.152030-1-richard.henderson@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 accel/tcg/cputlb.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c
index 3b918fe018..2035b2ac0a 100644
--- a/accel/tcg/cputlb.c
+++ b/accel/tcg/cputlb.c
@@ -2174,7 +2174,7 @@ uint32_t cpu_ldl_be_mmu(CPUArchState *env, abi_ptr addr,
 uint64_t cpu_ldq_be_mmu(CPUArchState *env, abi_ptr addr,
                         MemOpIdx oi, uintptr_t ra)
 {
-    return cpu_load_helper(env, addr, oi, MO_BEUQ, helper_be_ldq_mmu);
+    return cpu_load_helper(env, addr, oi, ra, helper_be_ldq_mmu);
 }
 
 uint16_t cpu_ldw_le_mmu(CPUArchState *env, abi_ptr addr,
-- 
2.27.0

[PULL 7/8] softmmu: List CPU types again

[Thomas Huth]

From: Philippe Mathieu-Daudé <f4bug@amsat.org>

Commit e0220bb5b2 made cpus.c target-agnostic but didn't notice
the cpu_list() function is only defined in target-specific code
in "cpu.h". Move list_cpus() declaration to "exec/cpu-common.h"
because this function is not softmmu-specific and can also be
used by user-mode, along with moving its implementation to cpu.c,
which is compiled per target.

Fixes: e0220bb5b2 ("softmmu: Build target-agnostic objects once")
Reported-by: Max Filippov <jcmvbkbc@gmail.com>
Suggested-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20220314140108.26222-1-philippe.mathieu.daude@gmail.com>
Tested-by: Max Filippov <jcmvbkbc@gmail.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 cpu.c                     | 9 +++++++++
 include/exec/cpu-common.h | 2 ++
 include/sysemu/cpus.h     | 2 --
 softmmu/cpus.c            | 8 --------
 4 files changed, 11 insertions(+), 10 deletions(-)

diff --git a/cpu.c b/cpu.c
index d564886149..be1f8b074c 100644
--- a/cpu.c
+++ b/cpu.c
@@ -35,6 +35,7 @@
 #include "sysemu/tcg.h"
 #include "sysemu/kvm.h"
 #include "sysemu/replay.h"
+#include "exec/cpu-common.h"
 #include "exec/exec-all.h"
 #include "exec/translate-all.h"
 #include "exec/log.h"
@@ -266,6 +267,14 @@ const char *parse_cpu_option(const char *cpu_option)
     return cpu_type;
 }
 
+void list_cpus(const char *optarg)
+{
+    /* XXX: implement xxx_cpu_list for targets that still miss it */
+#if defined(cpu_list)
+    cpu_list();
+#endif
+}
+
 #if defined(CONFIG_USER_ONLY)
 void tb_invalidate_phys_addr(target_ulong addr)
 {
diff --git a/include/exec/cpu-common.h b/include/exec/cpu-common.h
index 7f7b5943c7..50a7d2912e 100644
--- a/include/exec/cpu-common.h
+++ b/include/exec/cpu-common.h
@@ -158,4 +158,6 @@ int cpu_memory_rw_debug(CPUState *cpu, vaddr addr,
 /* vl.c */
 extern int singlestep;
 
+void list_cpus(const char *optarg);
+
 #endif /* CPU_COMMON_H */
diff --git a/include/sysemu/cpus.h b/include/sysemu/cpus.h
index 868f1192de..b5c87d48b3 100644
--- a/include/sysemu/cpus.h
+++ b/include/sysemu/cpus.h
@@ -55,6 +55,4 @@ extern int smp_cores;
 extern int smp_threads;
 #endif
 
-void list_cpus(const char *optarg);
-
 #endif
diff --git a/softmmu/cpus.c b/softmmu/cpus.c
index e1d84c8ccb..7b75bb66d5 100644
--- a/softmmu/cpus.c
+++ b/softmmu/cpus.c
@@ -728,14 +728,6 @@ int vm_stop_force_state(RunState state)
     }
 }
 
-void list_cpus(const char *optarg)
-{
-    /* XXX: implement xxx_cpu_list for targets that still miss it */
-#if defined(cpu_list)
-    cpu_list();
-#endif
-}
-
 void qmp_memsave(int64_t addr, int64_t size, const char *filename,
                  bool has_cpu, int64_t cpu_index, Error **errp)
 {
-- 
2.27.0

[PULL 8/8] meson: Update to version 0.61.3

[Thomas Huth]

Meson 0.61.3 contains an important fix which helps to see the
output of failed qemu-iotests on the console again:
https://gitlab.com/qemu-project/meson/-/commit/7534cf34f83b9c43

Acked-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20220315093220.407895-1-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 configure | 2 +-
 meson     | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/configure b/configure
index 886000346a..8e3c41ff92 100755
--- a/configure
+++ b/configure
@@ -1286,7 +1286,7 @@ python_version=$($python -c 'import sys; print("%d.%d.%d" % (sys.version_info[0]
 python="$python -B"
 
 if test -z "$meson"; then
-    if test "$explicit_python" = no && has meson && version_ge "$(meson --version)" 0.59.3; then
+    if test "$explicit_python" = no && has meson && version_ge "$(meson --version)" 0.61.3; then
         meson=meson
     elif test $git_submodules_action != 'ignore' ; then
         meson=git
diff --git a/meson b/meson
index 12f9f04ba0..5cf5575a7c 160000
--- a/meson
+++ b/meson
@@ -1 +1 @@
-Subproject commit 12f9f04ba0decfda425dbbf9a501084c153a2d18
+Subproject commit 5cf5575a7c76746935dcd9a9e380803c85023c04
-- 
2.27.0

Re: [PULL 0/8] s390x and misc fixes

[Peter Maydell]

On Tue, 15 Mar 2022 at 11:20, Thomas Huth <thuth@redhat.com> wrote:
>
>  Hi Peter!
>
> The following changes since commit 352998df1c53b366413690d95b35f76d0721ebed:
>
>   Merge tag 'i2c-20220314' of https://github.com/philmd/qemu into staging (2022-03-14 14:39:33 +0000)
>
> are available in the Git repository at:
>
>   https://gitlab.com/thuth/qemu.git tags/pull-request-2022-03-15
>
> for you to fetch changes up to 36149534792dcf07a3c59867f967eaee23ab906c:
>
>   meson: Update to version 0.61.3 (2022-03-15 10:32:36 +0100)
>
> ----------------------------------------------------------------
> * Fixes for s390x branch instruction emulation
> * Fixes for the tests/avocado/boot_linux.py:BootLinuxS390X test
> * Fix for "-cpu help" output
> * Bump meson to 0.61.3 to fix stderr log of the iotests
>
> ----------------------------------------------------------------

This results in every "Linking" step on my macos box producing the
warning:

ld: warning: directory not found for option
'-Lns/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/12.0.0'

Obvious suspect here is the new meson version.

thanks
-- PMM

Re: [PULL 0/8] s390x and misc fixes

[Peter Maydell]

On Tue, 15 Mar 2022 at 18:58, Peter Maydell <peter.maydell@linaro.org> wrote:
>
> On Tue, 15 Mar 2022 at 11:20, Thomas Huth <thuth@redhat.com> wrote:
> >
> >  Hi Peter!
> >
> > The following changes since commit 352998df1c53b366413690d95b35f76d0721ebed:
> >
> >   Merge tag 'i2c-20220314' of https://github.com/philmd/qemu into staging (2022-03-14 14:39:33 +0000)
> >
> > are available in the Git repository at:
> >
> >   https://gitlab.com/thuth/qemu.git tags/pull-request-2022-03-15
> >
> > for you to fetch changes up to 36149534792dcf07a3c59867f967eaee23ab906c:
> >
> >   meson: Update to version 0.61.3 (2022-03-15 10:32:36 +0100)
> >
> > ----------------------------------------------------------------
> > * Fixes for s390x branch instruction emulation
> > * Fixes for the tests/avocado/boot_linux.py:BootLinuxS390X test
> > * Fix for "-cpu help" output
> > * Bump meson to 0.61.3 to fix stderr log of the iotests
> >
> > ----------------------------------------------------------------
>
> This results in every "Linking" step on my macos box producing the
> warning:
>
> ld: warning: directory not found for option
> '-Lns/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/12.0.0'
>
> Obvious suspect here is the new meson version.

Also, after rolling this merge attempt back, older meson barfs
on whatever the new one left behind:


[0/1] Regenerating build files.
Traceback (most recent call last):
  File "/Users/pm215/src/qemu-for-merges/meson/mesonbuild/mesonmain.py",
line 228, in run
    return options.run_func(options)
  File "/Users/pm215/src/qemu-for-merges/meson/mesonbuild/msetup.py",
line 281, in run
    app.generate()
  File "/Users/pm215/src/qemu-for-merges/meson/mesonbuild/msetup.py",
line 177, in generate
    env = environment.Environment(self.source_dir, self.build_dir, self.options)
  File "/Users/pm215/src/qemu-for-merges/meson/mesonbuild/environment.py",
line 462, in __init__
    self.coredata = coredata.load(self.get_build_dir())  # type:
coredata.CoreData
  File "/Users/pm215/src/qemu-for-merges/meson/mesonbuild/coredata.py",
line 1003, in load
    obj = pickle.load(f)
  File "/Users/pm215/src/qemu-for-merges/meson/mesonbuild/mesonlib/universal.py",
line 2076, in __setstate__
    self.__init__(**state)  # type: ignore
TypeError: __init__() got an unexpected keyword argument 'module'
FAILED: build.ninja
/usr/local/opt/python@3.9/bin/python3.9
/Users/pm215/src/qemu-for-merges/meson/meson.py --internal regenerate
/Users/pm215/src/qemu-for-merges
/Users/pm215/src/qemu-for-merges/build/all --backend ninja
ninja: error: rebuilding 'build.ninja': subcommand failed
/usr/local/bin/ninja  build.ninja && touch build.ninja.stamp
  GIT     ui/keycodemapdb meson tests/fp/berkeley-testfloat-3
tests/fp/berkeley-softfloat-3 dtc capstone slirp
[0/1] Regenerating build files.
Traceback (most recent call last):
  File "/Users/pm215/src/qemu-for-merges/meson/mesonbuild/mesonmain.py",
line 228, in run
    return options.run_func(options)
  File "/Users/pm215/src/qemu-for-merges/meson/mesonbuild/msetup.py",
line 281, in run
    app.generate()
  File "/Users/pm215/src/qemu-for-merges/meson/mesonbuild/msetup.py",
line 177, in generate
    env = environment.Environment(self.source_dir, self.build_dir, self.options)
  File "/Users/pm215/src/qemu-for-merges/meson/mesonbuild/environment.py",
line 462, in __init__
    self.coredata = coredata.load(self.get_build_dir())  # type:
coredata.CoreData
  File "/Users/pm215/src/qemu-for-merges/meson/mesonbuild/coredata.py",
line 1003, in load
    obj = pickle.load(f)
  File "/Users/pm215/src/qemu-for-merges/meson/mesonbuild/mesonlib/universal.py",
line 2076, in __setstate__
    self.__init__(**state)  # type: ignore
TypeError: __init__() got an unexpected keyword argument 'module'
FAILED: build.ninja


meson ought to be smart enough to spot that it's got data from an
incompatible version and just discard its cache rather than
choking on it.

thanks
-- PMM

Re: [PULL 0/8] s390x and misc fixes

[Thomas Huth]

On 15/03/2022 20.30, Peter Maydell wrote:
> On Tue, 15 Mar 2022 at 18:58, Peter Maydell <peter.maydell@linaro.org> wrote:
>>
>> On Tue, 15 Mar 2022 at 11:20, Thomas Huth <thuth@redhat.com> wrote:
>>>
>>>   Hi Peter!
>>>
>>> The following changes since commit 352998df1c53b366413690d95b35f76d0721ebed:
>>>
>>>    Merge tag 'i2c-20220314' of https://github.com/philmd/qemu into staging (2022-03-14 14:39:33 +0000)
>>>
>>> are available in the Git repository at:
>>>
>>>    https://gitlab.com/thuth/qemu.git tags/pull-request-2022-03-15
>>>
>>> for you to fetch changes up to 36149534792dcf07a3c59867f967eaee23ab906c:
>>>
>>>    meson: Update to version 0.61.3 (2022-03-15 10:32:36 +0100)
>>>
>>> ----------------------------------------------------------------
>>> * Fixes for s390x branch instruction emulation
>>> * Fixes for the tests/avocado/boot_linux.py:BootLinuxS390X test
>>> * Fix for "-cpu help" output
>>> * Bump meson to 0.61.3 to fix stderr log of the iotests
>>>
>>> ----------------------------------------------------------------
>>
>> This results in every "Linking" step on my macos box producing the
>> warning:
>>
>> ld: warning: directory not found for option
>> '-Lns/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/12.0.0'
>>
>> Obvious suspect here is the new meson version.
> 
> Also, after rolling this merge attempt back, older meson barfs
> on whatever the new one left behind:
> 
> 
> [0/1] Regenerating build files.
> Traceback (most recent call last):
>    File "/Users/pm215/src/qemu-for-merges/meson/mesonbuild/mesonmain.py",
> line 228, in run
>      return options.run_func(options)
>    File "/Users/pm215/src/qemu-for-merges/meson/mesonbuild/msetup.py",
> line 281, in run
>      app.generate()
>    File "/Users/pm215/src/qemu-for-merges/meson/mesonbuild/msetup.py",
> line 177, in generate
>      env = environment.Environment(self.source_dir, self.build_dir, self.options)
>    File "/Users/pm215/src/qemu-for-merges/meson/mesonbuild/environment.py",
> line 462, in __init__
>      self.coredata = coredata.load(self.get_build_dir())  # type:
> coredata.CoreData
>    File "/Users/pm215/src/qemu-for-merges/meson/mesonbuild/coredata.py",
> line 1003, in load
>      obj = pickle.load(f)
>    File "/Users/pm215/src/qemu-for-merges/meson/mesonbuild/mesonlib/universal.py",
> line 2076, in __setstate__
>      self.__init__(**state)  # type: ignore
> TypeError: __init__() got an unexpected keyword argument 'module'
> FAILED: build.ninja
> /usr/local/opt/python@3.9/bin/python3.9
> /Users/pm215/src/qemu-for-merges/meson/meson.py --internal regenerate
> /Users/pm215/src/qemu-for-merges
> /Users/pm215/src/qemu-for-merges/build/all --backend ninja
> ninja: error: rebuilding 'build.ninja': subcommand failed
> /usr/local/bin/ninja  build.ninja && touch build.ninja.stamp
>    GIT     ui/keycodemapdb meson tests/fp/berkeley-testfloat-3
> tests/fp/berkeley-softfloat-3 dtc capstone slirp
> [0/1] Regenerating build files.
> Traceback (most recent call last):
>    File "/Users/pm215/src/qemu-for-merges/meson/mesonbuild/mesonmain.py",
> line 228, in run
>      return options.run_func(options)
>    File "/Users/pm215/src/qemu-for-merges/meson/mesonbuild/msetup.py",
> line 281, in run
>      app.generate()
>    File "/Users/pm215/src/qemu-for-merges/meson/mesonbuild/msetup.py",
> line 177, in generate
>      env = environment.Environment(self.source_dir, self.build_dir, self.options)
>    File "/Users/pm215/src/qemu-for-merges/meson/mesonbuild/environment.py",
> line 462, in __init__
>      self.coredata = coredata.load(self.get_build_dir())  # type:
> coredata.CoreData
>    File "/Users/pm215/src/qemu-for-merges/meson/mesonbuild/coredata.py",
> line 1003, in load
>      obj = pickle.load(f)
>    File "/Users/pm215/src/qemu-for-merges/meson/mesonbuild/mesonlib/universal.py",
> line 2076, in __setstate__
>      self.__init__(**state)  # type: ignore
> TypeError: __init__() got an unexpected keyword argument 'module'
> FAILED: build.ninja
> 
> 
> meson ought to be smart enough to spot that it's got data from an
> incompatible version and just discard its cache rather than
> choking on it.

Ok, I'll respin without the meson update.

Question is: Do we now want to revert the TAPification of the iotests for 
7.0? I guess so, otherwise most people won't see the output of failed tests 
when doing "make check-block" ...

  Thomas

Re: [PULL 0/8] s390x and misc fixes

[Paolo Bonzini]

On 3/15/22 20:30, Peter Maydell wrote:
> coredata.CoreData
>    File "/Users/pm215/src/qemu-for-merges/meson/mesonbuild/coredata.py",
> line 1003, in load
>      obj = pickle.load(f)
>    File "/Users/pm215/src/qemu-for-merges/meson/mesonbuild/mesonlib/universal.py",
> line 2076, in __setstate__
>      self.__init__(**state)  # type: ignore
> TypeError: __init__() got an unexpected keyword argument 'module'
> FAILED: build.ninja
> 
> 
> meson ought to be smart enough to spot that it's got data from an
> incompatible version and just discard its cache rather than
> choking on it.

It tries, but it's playing whack-a-mole with Python:

     except (pickle.UnpicklingError, EOFError):
         raise MesonException(load_fail_msg)
     except (ModuleNotFoundError, AttributeError):
         raise MesonException(
             f"Coredata file {filename!r} references functions or classes that don't "
             "exist. This probably means that it was generated with an old "
             "version of meson.")

I'll submit a patch to add TypeError.

Paolo

Re: [PULL 0/8] s390x and misc fixes

[Paolo Bonzini]

On 3/15/22 19:58, Peter Maydell wrote:
> On Tue, 15 Mar 2022 at 11:20, Thomas Huth <thuth@redhat.com> wrote:
>>
>>   Hi Peter!
>>
>> The following changes since commit 352998df1c53b366413690d95b35f76d0721ebed:
>>
>>    Merge tag 'i2c-20220314' of https://github.com/philmd/qemu into staging (2022-03-14 14:39:33 +0000)
>>
>> are available in the Git repository at:
>>
>>    https://gitlab.com/thuth/qemu.git tags/pull-request-2022-03-15
>>
>> for you to fetch changes up to 36149534792dcf07a3c59867f967eaee23ab906c:
>>
>>    meson: Update to version 0.61.3 (2022-03-15 10:32:36 +0100)
>>
>> ----------------------------------------------------------------
>> * Fixes for s390x branch instruction emulation
>> * Fixes for the tests/avocado/boot_linux.py:BootLinuxS390X test
>> * Fix for "-cpu help" output
>> * Bump meson to 0.61.3 to fix stderr log of the iotests
>>
>> ----------------------------------------------------------------
> 
> This results in every "Linking" step on my macos box producing the
> warning:
> 
> ld: warning: directory not found for option
> '-Lns/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/12.0.0'
> 
> Obvious suspect here is the new meson version.
> 

Can you send the meson-logs/meson-log.txt and build.ninja files?  Thanks,

Paolo

Re: [PULL 0/8] s390x and misc fixes

[Peter Maydell]

On Wed, 16 Mar 2022 at 12:18, Paolo Bonzini <pbonzini@redhat.com> wrote:
>
> On 3/15/22 19:58, Peter Maydell wrote:
> > On Tue, 15 Mar 2022 at 11:20, Thomas Huth <thuth@redhat.com> wrote:
> > This results in every "Linking" step on my macos box producing the
> > warning:
> >
> > ld: warning: directory not found for option
> > '-Lns/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/12.0.0'
> >
> > Obvious suspect here is the new meson version.
> >
>
> Can you send the meson-logs/meson-log.txt and build.ninja files?

https://people.linaro.org/~peter.maydell/meson-log.txt
https://people.linaro.org/~peter.maydell/build.ninja

Something seems to have truncated a path somewhere. You can see
in meson-log.txt that it uses --print-search-dirs and gets

 programs: =/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
libraries: =/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/12.0.0

but in build.ninja the LINK_ARGS start
-Lns/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/12.0.0

with the "/Applicatio" part removed somehow.

I'm not sure why we're passing this explicitly to the linker anyway,
though.

-- PMM

Re: [PULL 0/8] s390x and misc fixes

[Paolo Bonzini]

On 3/16/22 14:19, Peter Maydell wrote:
> I'm not sure why we're passing this explicitly to the linker anyway,
> though.

According to the commit message the idea is to add libc++ automatically 
when you have C++ sources, libgfortran when you have Fortran sources 
etc. independent of the language that you're using for the final link.

In other words it removes the need to specify a linker driver for 
mixed-language programs (and allows some other combinations such as C++ 
and Fortran together).

It would be better if it worked though...

Paolo

Re: [PULL 0/8] s390x and misc fixes

[Paolo Bonzini]

On 3/15/22 19:58, Peter Maydell wrote:
> This results in every "Linking" step on my macos box producing the
> warning:
> 
> ld: warning: directory not found for option
> '-Lns/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/12.0.0'

This was reported already back in January with Meson 0.59:

https://www.mail-archive.com/qemu-devel@nongnu.org/msg861435.html

> 1. Linker has corrupted paths to clang directory (happens on all macOS 
> versions).
> 
> Monterey:
> 
> [732/737] Linking target qemu-system-mips-unsigned
> ld: warning: directory not found for option 
> '-Lns/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/13.0.0'
> [733/737] Linking target qemu-system-mips64-unsigned
> ld: warning: directory not found for option 
> '-Lns/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/13.0.0'
> [737/737] Generating qemu-system-mips64 with a custom command
> 
> Catalina:
> 
> ld: warning: directory not found for option 
> '-Lveloper/CommandLineTools/usr/lib/clang/11.0.0'
> [102/105] Linking target qemu-system-or1k-unsigned
> ld: warning: directory not found for option 
> '-Lveloper/CommandLineTools/usr/lib/clang/11.0.0'
> [104/105] Linking target qemu-system-ppc-unsigned
> ld: warning: directory not found for option 
> '-Lveloper/CommandLineTools/usr/lib/clang/11.0.0'
> [105/105] Generating qemu-system-ppc with a custom command

Roman, was this ever debugged?  Is the broken path to the clang 
directory included in build.ninja or is this a clang bug that somehow 
has been latent so far?

Paolo