[PULL 0/9] final misc fixes for 5.1-rc0

[PULL 0/9] final misc fixes for 5.1-rc0

[Alex Bennée]

The following changes since commit 20c1df5476e1e9b5d3f5b94f9f3ce01d21f14c46:

  Merge remote-tracking branch 'remotes/kraxel/tags/fixes-20200713-pull-request' into staging (2020-07-13 16:58:44 +0100)

are available in the Git repository at:

  https://github.com/stsquad/qemu.git tags/pull-misc-for-rc0-140720-1

for you to fetch changes up to 4e6400c97024c2660b6db8aab1f0677744449f36:

  plugins: expand the bb plugin to be thread safe and track per-cpu (2020-07-14 09:08:25 +0100)

----------------------------------------------------------------
Final fixes for 5.1-rc0

  - minor documentation nit
  - clean all deps on re(configure)
  - docker.py bootstrap fixes
  - tweak containers.yml wildcards
  - fix float16 nan detection
  - conditional use of -Wpsabi
  - fix missing iotlb data for plugins
  - proper locking for helper based bb count

----------------------------------------------------------------
Alex Bennée (6):
      docs/devel: fix grammar in multi-thread-tcg
      configure: remove all dependencies on a (re)configure
      docker.py: fix fetching of FROM layers
      tests/plugins: don't unconditionally add -Wpsabi
      cputlb: ensure we save the IOTLB data in case of reset
      plugins: expand the bb plugin to be thread safe and track per-cpu

LIU Zhiwei (1):
      fpu/softfloat: fix up float16 nan recognition

Thomas Huth (2):
      tests/docker: Remove the libssh workaround from the ubuntu 20.04 image
      gitlab-ci/containers: Add missing wildcard where we should look for changes

 docs/devel/multi-thread-tcg.rst            |  2 +-
 configure                                  | 18 +++---
 include/hw/core/cpu.h                      | 16 +++++
 include/qemu/typedefs.h                    |  1 +
 accel/tcg/cputlb.c                         | 38 +++++++++++-
 fpu/softfloat-specialize.inc.c             |  4 +-
 tests/plugin/bb.c                          | 97 ++++++++++++++++++++++++++----
 .gitlab-ci.d/containers.yml                |  1 +
 tests/docker/docker.py                     | 16 +++--
 tests/docker/dockerfiles/ubuntu2004.docker |  3 -
 tests/plugin/Makefile                      | 22 +++++--
 11 files changed, 178 insertions(+), 40 deletions(-)

-- 
2.20.1

[PULL 1/9] docs/devel: fix grammar in multi-thread-tcg

[Alex Bennée]

Review comment came just too late ;-)

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20200713200415.26214-9-alex.bennee@linaro.org>

diff --git a/docs/devel/multi-thread-tcg.rst b/docs/devel/multi-thread-tcg.rst
index 42158b77c7..21483870db 100644
--- a/docs/devel/multi-thread-tcg.rst
+++ b/docs/devel/multi-thread-tcg.rst
@@ -107,7 +107,7 @@ including:
 
   - debugging operations (breakpoint insertion/removal)
   - some CPU helper functions
-  - linux-user spawning it's first thread
+  - linux-user spawning its first thread
 
 This is done with the async_safe_run_on_cpu() mechanism to ensure all
 vCPUs are quiescent when changes are being made to shared global
-- 
2.20.1

[PULL 2/9] configure: remove all dependencies on a (re)configure

[Alex Bennée]

The previous code was brittle and missed cases such as the mipn32
variants which for some reason has the 64 bit syscalls. This leads to
a number of binary targets having deps lines like:

  all.clang-sanitizer/mipsn32el-linux-user/linux-user/signal.d
  140:  /home/alex/lsrc/qemu.git/linux-user/mips64/syscall_nr.h \
  455:/home/alex/lsrc/qemu.git/linux-user/mips64/syscall_nr.h:

  all.clang-sanitizer/mipsn32el-linux-user/linux-user/syscall.d
  146:  /home/alex/lsrc/qemu.git/linux-user/mips64/syscall_nr.h \
  485:/home/alex/lsrc/qemu.git/linux-user/mips64/syscall_nr.h:

which in turn would trigger the re-generation of syscall_nr.h in the
source tree (thanks to generic %/syscall_nr.h rules). The previous
code attempts to clean it out but misses edge cases but fails.

After spending a day trying to understand how this was happening I'm
unconvinced that there are not other such breakages possible with this
"caching". As we add more auto-generated code to the build it is likely
to trip up again. Apply a hammer to the problem.

Fixes: 91e5998f18 (which fixes 5f29856b852d and 4d6a835dea47)
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200713200415.26214-2-alex.bennee@linaro.org>

diff --git a/configure b/configure
index bc3b9ad931..e1de2f5b24 100755
--- a/configure
+++ b/configure
@@ -1955,23 +1955,20 @@ EOF
 exit 0
 fi
 
-# Remove old dependency files to make sure that they get properly regenerated
-rm -f */config-devices.mak.d
-
 # Remove syscall_nr.h to be sure they will be regenerated in the build
 # directory, not in the source directory
 for arch in alpha hppa m68k xtensa sh4 microblaze arm ppc s390x sparc sparc64 \
     i386 x86_64 mips mips64 ; do
     # remove the file if it has been generated in the source directory
     rm -f "${source_path}/linux-user/${arch}/syscall_nr.h"
-    # remove the dependency files
-    for target in ${arch}*-linux-user ; do
-        test -d "${target}" && find "${target}" -type f -name "*.d" \
-             -exec grep -q "${source_path}/linux-user/${arch}/syscall_nr.h" {} \; \
-             -print | while read file ; do rm "${file}" "${file%.d}.o" ; done
-    done
 done
 
+# Clean out all old dependency files. As more files are generated we
+# run the risk of old dependencies triggering generation in the wrong
+# places. Previous brittle attempts to be surgical tend to miss edge
+# cases leading to wasted time and much confusion.
+find -type f -name "*.d" -exec rm -f {} \;
+
 if test -z "$python"
 then
     error_exit "Python not found. Use --python=/path/to/python"
-- 
2.20.1

[PULL 3/9] tests/docker: Remove the libssh workaround from the ubuntu 20.04 image

[Alex Bennée]

From: Thomas Huth <thuth@redhat.com>

The libssh problem only exists in Ubuntu 18.04 - we can enable it
in 20.04 again.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20200713185237.9419-1-thuth@redhat.com>
Message-Id: <20200713200415.26214-3-alex.bennee@linaro.org>

diff --git a/tests/docker/dockerfiles/ubuntu2004.docker b/tests/docker/dockerfiles/ubuntu2004.docker
index f7aac840bf..8d10934a2a 100644
--- a/tests/docker/dockerfiles/ubuntu2004.docker
+++ b/tests/docker/dockerfiles/ubuntu2004.docker
@@ -65,9 +65,6 @@ RUN apt-get update && \
 RUN dpkg -l $PACKAGES | sort > /packages.txt
 ENV FEATURES clang tsan pyyaml sdl2
 
-# https://bugs.launchpad.net/qemu/+bug/1838763
-ENV QEMU_CONFIGURE_OPTS --disable-libssh
-
 # Apply patch https://reviews.llvm.org/D75820
 # This is required for TSan in clang-10 to compile with QEMU.
 RUN sed -i 's/^const/static const/g' /usr/lib/llvm-10/lib/clang/10.0.0/include/sanitizer/tsan_interface.h
-- 
2.20.1

[PULL 4/9] docker.py: fix fetching of FROM layers

[Alex Bennée]

This worked on a system that was already bootstrapped because the
stage 2 images already existed even if they wouldn't be used. What we
should have pulled down was the FROM line containers first because
building on gitlab doesn't have the advantage of using our build
system to build the pre-requisite bits.

We still pull the image we want to build just in case we can use the
cached data.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20200713200415.26214-4-alex.bennee@linaro.org>

diff --git a/tests/docker/docker.py b/tests/docker/docker.py
index 2d67bbd15a..c9f20d8d09 100755
--- a/tests/docker/docker.py
+++ b/tests/docker/docker.py
@@ -306,14 +306,18 @@ class Docker(object):
         checksum = _text_checksum(_dockerfile_preprocess(dockerfile))
 
         if registry is not None:
-            # see if we can fetch a cache copy, may fail...
-            pull_args = ["pull", "%s/%s" % (registry, tag)]
-            if self._do(pull_args, quiet=quiet) == 0:
+            sources = re.findall("FROM qemu\/(.*)", dockerfile)
+            # Fetch any cache layers we can, may fail
+            for s in sources:
+                pull_args = ["pull", "%s/qemu/%s" % (registry, s)]
+                if self._do(pull_args, quiet=quiet) != 0:
+                    registry = None
+                    break
+            # Make substitutions
+            if registry is not None:
                 dockerfile = dockerfile.replace("FROM qemu/",
                                                 "FROM %s/qemu/" %
                                                 (registry))
-            else:
-                registry = None
 
         tmp_df = tempfile.NamedTemporaryFile(mode="w+t",
                                              encoding='utf-8',
@@ -339,6 +343,8 @@ class Docker(object):
             build_args += ["--build-arg", "BUILDKIT_INLINE_CACHE=1"]
 
         if registry is not None:
+            pull_args = ["pull", "%s/%s" % (registry, tag)]
+            self._do(pull_args, quiet=quiet)
             cache = "%s/%s" % (registry, tag)
             build_args += ["--cache-from", cache]
         build_args += argv
-- 
2.20.1

[PULL 5/9] gitlab-ci/containers: Add missing wildcard where we should look for changes

[Alex Bennée]

From: Thomas Huth <thuth@redhat.com>

The tests/docker/* wildcard seems to only match the files that are directly
in the tests/docker folder - but changes to the files in the directory
tests/docker/dockerfiles are currently ignored. Seems like we need a
separate entry to match the files in that folder. With this wildcard added,
the stages now get re-run successfully when something in the dockerfiles
has been changed.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200713182235.30379-1-thuth@redhat.com>

diff --git a/.gitlab-ci.d/containers.yml b/.gitlab-ci.d/containers.yml
index f3c0ca4d61..8c89efeb6d 100644
--- a/.gitlab-ci.d/containers.yml
+++ b/.gitlab-ci.d/containers.yml
@@ -24,6 +24,7 @@
     - changes:
       - .gitlab-ci.d/containers.yml
       - tests/docker/*
+      - tests/docker/dockerfiles/*
     - if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH'
     - if: '$CI_COMMIT_REF_NAME == "testing/next"'
 
-- 
2.20.1

[PULL 6/9] fpu/softfloat: fix up float16 nan recognition

[Alex Bennée]

From: LIU Zhiwei <zhiwei_liu@c-sky.com>

Signed-off-by: LIU Zhiwei <zhiwei_liu@c-sky.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20200712234521.3972-2-zhiwei_liu@c-sky.com>
Message-Id: <20200713200415.26214-5-alex.bennee@linaro.org>

diff --git a/fpu/softfloat-specialize.inc.c b/fpu/softfloat-specialize.inc.c
index 44f5b661f8..034d18199c 100644
--- a/fpu/softfloat-specialize.inc.c
+++ b/fpu/softfloat-specialize.inc.c
@@ -254,7 +254,7 @@ bool float16_is_quiet_nan(float16 a_, float_status *status)
     if (snan_bit_is_one(status)) {
         return (((a >> 9) & 0x3F) == 0x3E) && (a & 0x1FF);
     } else {
-        return ((a & ~0x8000) >= 0x7C80);
+        return ((a >> 9) & 0x3F) == 0x3F;
     }
 #endif
 }
@@ -271,7 +271,7 @@ bool float16_is_signaling_nan(float16 a_, float_status *status)
 #else
     uint16_t a = float16_val(a_);
     if (snan_bit_is_one(status)) {
-        return ((a & ~0x8000) >= 0x7C80);
+        return ((a >> 9) & 0x3F) == 0x3F;
     } else {
         return (((a >> 9) & 0x3F) == 0x3E) && (a & 0x1FF);
     }
-- 
2.20.1

[PULL 7/9] tests/plugins: don't unconditionally add -Wpsabi

[Alex Bennée]

Not all compilers support the -Wpsabi (clang-9 in my case). To handle
this gracefully we pare back the shared build machinery so the
Makefile is relatively "standalone". We still take advantage of
config-host.mak as configure has done a bunch of probing for us but
that is it.

Fixes: bac8d222a
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20200713200415.26214-6-alex.bennee@linaro.org>

diff --git a/configure b/configure
index e1de2f5b24..08eaa99d19 100755
--- a/configure
+++ b/configure
@@ -7112,6 +7112,9 @@ echo "GIT_UPDATE=$git_update" >> $config_host_mak
 
 echo "ARCH=$ARCH" >> $config_host_mak
 
+echo "GLIB_CFLAGS=$glib_cflags" >> $config_host_mak
+echo "GLIB_LDFLAGS=$glib_ldflags" >> $config_host_mak
+
 if test "$default_devices" = "yes" ; then
   echo "CONFIG_MINIKCONF_MODE=--defconfig" >> $config_host_mak
 else
diff --git a/tests/plugin/Makefile b/tests/plugin/Makefile
index 3a50451428..e9348fde4a 100644
--- a/tests/plugin/Makefile
+++ b/tests/plugin/Makefile
@@ -1,9 +1,16 @@
+# -*- Mode: makefile -*-
+#
+# This Makefile example is fairly independent from the main makefile
+# so users can take and adapt it for their build. We only really
+# include config-host.mak so we don't have to repeat probing for
+# cflags that the main configure has already done for us.
+#
+
 BUILD_DIR := $(CURDIR)/../..
 
 include $(BUILD_DIR)/config-host.mak
-include $(SRC_PATH)/rules.mak
 
-$(call set-vpath, $(SRC_PATH)/tests/plugin)
+VPATH += $(SRC_PATH)/tests/plugin
 
 NAMES :=
 NAMES += bb
@@ -17,11 +24,18 @@ NAMES += lockstep
 
 SONAMES := $(addsuffix .so,$(addprefix lib,$(NAMES)))
 
-QEMU_CFLAGS += -fPIC -Wpsabi
-QEMU_CFLAGS += -I$(SRC_PATH)/include/qemu
+# The main QEMU uses Glib extensively so it's perfectly fine to use it
+# in plugins (which many example do).
+CFLAGS = $(GLIB_CFLAGS)
+CFLAGS += -fPIC
+CFLAGS += $(if $(findstring no-psabi,$(QEMU_CFLAGS)),-Wpsabi)
+CFLAGS += -I$(SRC_PATH)/include/qemu
 
 all: $(SONAMES)
 
+%.o: %.c
+	$(CC) $(CFLAGS) -c -o $@ $<
+
 lib%.so: %.o
 	$(CC) -shared -Wl,-soname,$@ -o $@ $^ $(LDLIBS)
 
-- 
2.20.1

[PULL 8/9] cputlb: ensure we save the IOTLB data in case of reset

[Alex Bennée]

Any write to a device might cause a re-arrangement of memory
triggering a TLB flush and potential re-size of the TLB invalidating
previous entries. This would cause users of qemu_plugin_get_hwaddr()
to see the warning:

  invalid use of qemu_plugin_get_hwaddr

because of the failed tlb_lookup which should always succeed. To
prevent this we save the IOTLB data in case it is later needed by a
plugin doing a lookup.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20200713200415.26214-7-alex.bennee@linaro.org>

diff --git a/include/hw/core/cpu.h b/include/hw/core/cpu.h
index 5542577d2b..8f145733ce 100644
--- a/include/hw/core/cpu.h
+++ b/include/hw/core/cpu.h
@@ -259,6 +259,18 @@ struct CPUWatchpoint {
     QTAILQ_ENTRY(CPUWatchpoint) entry;
 };
 
+#ifdef CONFIG_PLUGIN
+/*
+ * For plugins we sometime need to save the resolved iotlb data before
+ * the memory regions get moved around  by io_writex.
+ */
+typedef struct SavedIOTLB {
+    hwaddr addr;
+    MemoryRegionSection *section;
+    hwaddr mr_offset;
+} SavedIOTLB;
+#endif
+
 struct KVMState;
 struct kvm_run;
 
@@ -417,7 +429,11 @@ struct CPUState {
 
     DECLARE_BITMAP(plugin_mask, QEMU_PLUGIN_EV_MAX);
 
+#ifdef CONFIG_PLUGIN
     GArray *plugin_mem_cbs;
+    /* saved iotlb data from io_writex */
+    SavedIOTLB saved_iotlb;
+#endif
 
     /* TODO Move common fields from CPUArchState here. */
     int cpu_index;
diff --git a/include/qemu/typedefs.h b/include/qemu/typedefs.h
index 15f5047bf1..427027a970 100644
--- a/include/qemu/typedefs.h
+++ b/include/qemu/typedefs.h
@@ -116,6 +116,7 @@ typedef struct QObject QObject;
 typedef struct QString QString;
 typedef struct RAMBlock RAMBlock;
 typedef struct Range Range;
+typedef struct SavedIOTLB SavedIOTLB;
 typedef struct SHPCDevice SHPCDevice;
 typedef struct SSIBus SSIBus;
 typedef struct VirtIODevice VirtIODevice;
diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c
index 1e815357c7..d370aedb47 100644
--- a/accel/tcg/cputlb.c
+++ b/accel/tcg/cputlb.c
@@ -1073,6 +1073,24 @@ static uint64_t io_readx(CPUArchState *env, CPUIOTLBEntry *iotlbentry,
     return val;
 }
 
+/*
+ * Save a potentially trashed IOTLB entry for later lookup by plugin.
+ *
+ * We also need to track the thread storage address because the RCU
+ * cleanup that runs when we leave the critical region (the current
+ * execution) is actually in a different thread.
+ */
+static void save_iotlb_data(CPUState *cs, hwaddr addr,
+                            MemoryRegionSection *section, hwaddr mr_offset)
+{
+#ifdef CONFIG_PLUGIN
+    SavedIOTLB *saved = &cs->saved_iotlb;
+    saved->addr = addr;
+    saved->section = section;
+    saved->mr_offset = mr_offset;
+#endif
+}
+
 static void io_writex(CPUArchState *env, CPUIOTLBEntry *iotlbentry,
                       int mmu_idx, uint64_t val, target_ulong addr,
                       uintptr_t retaddr, MemOp op)
@@ -1092,6 +1110,12 @@ static void io_writex(CPUArchState *env, CPUIOTLBEntry *iotlbentry,
     }
     cpu->mem_io_pc = retaddr;
 
+    /*
+     * The memory_region_dispatch may trigger a flush/resize
+     * so for plugins we save the iotlb_data just in case.
+     */
+    save_iotlb_data(cpu, iotlbentry->addr, section, mr_offset);
+
     if (mr->global_locking && !qemu_mutex_iothread_locked()) {
         qemu_mutex_lock_iothread();
         locked = true;
@@ -1381,8 +1405,11 @@ void *tlb_vaddr_to_host(CPUArchState *env, abi_ptr addr,
  * in the softmmu lookup code (or helper). We don't handle re-fills or
  * checking the victim table. This is purely informational.
  *
- * This should never fail as the memory access being instrumented
- * should have just filled the TLB.
+ * This almost never fails as the memory access being instrumented
+ * should have just filled the TLB. The one corner case is io_writex
+ * which can cause TLB flushes and potential resizing of the TLBs
+ * loosing the information we need. In those cases we need to recover
+ * data from a copy of the io_tlb entry.
  */
 
 bool tlb_plugin_lookup(CPUState *cpu, target_ulong addr, int mmu_idx,
@@ -1406,8 +1433,13 @@ bool tlb_plugin_lookup(CPUState *cpu, target_ulong addr, int mmu_idx,
             data->v.ram.hostaddr = addr + tlbe->addend;
         }
         return true;
+    } else {
+        SavedIOTLB *saved = &cpu->saved_iotlb;
+        data->is_io = true;
+        data->v.io.section = saved->section;
+        data->v.io.offset = saved->mr_offset;
+        return true;
     }
-    return false;
 }
 
 #endif
-- 
2.20.1

[PULL 9/9] plugins: expand the bb plugin to be thread safe and track per-cpu

[Alex Bennée]

While there isn't any easy way to make the inline counts thread safe
we can ensure the callback based ones are. While we are at it we can
reduce introduce a new option ("idle") to dump a report of the current
bb and insn count each time a vCPU enters the idle state.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Robert Foley <robert.foley@linaro.org>
Cc: Dave Bort <dbort@dbort.com>

Message-Id: <20200713200415.26214-8-alex.bennee@linaro.org>

diff --git a/tests/plugin/bb.c b/tests/plugin/bb.c
index df19fd359d..e4cc7fdd6e 100644
--- a/tests/plugin/bb.c
+++ b/tests/plugin/bb.c
@@ -16,24 +16,67 @@
 
 QEMU_PLUGIN_EXPORT int qemu_plugin_version = QEMU_PLUGIN_VERSION;
 
-static uint64_t bb_count;
-static uint64_t insn_count;
+typedef struct {
+    GMutex lock;
+    int index;
+    uint64_t bb_count;
+    uint64_t insn_count;
+} CPUCount;
+
+/* Used by the inline & linux-user counts */
 static bool do_inline;
+static CPUCount inline_count;
+
+/* Dump running CPU total on idle? */
+static bool idle_report;
+static GPtrArray *counts;
+static int max_cpus;
+
+static void gen_one_cpu_report(CPUCount *count, GString *report)
+{
+    if (count->bb_count) {
+        g_string_append_printf(report, "CPU%d: "
+                               "bb's: %" PRIu64", insns: %" PRIu64 "\n",
+                               count->index,
+                               count->bb_count, count->insn_count);
+    }
+}
 
 static void plugin_exit(qemu_plugin_id_t id, void *p)
 {
-    g_autofree gchar *out = g_strdup_printf(
-        "bb's: %" PRIu64", insns: %" PRIu64 "\n",
-        bb_count, insn_count);
-    qemu_plugin_outs(out);
+    g_autoptr(GString) report = g_string_new("");
+
+    if (do_inline || !max_cpus) {
+        g_string_printf(report, "bb's: %" PRIu64", insns: %" PRIu64 "\n",
+                        inline_count.bb_count, inline_count.insn_count);
+    } else {
+        g_ptr_array_foreach(counts, (GFunc) gen_one_cpu_report, report);
+    }
+    qemu_plugin_outs(report->str);
+}
+
+static void vcpu_idle(qemu_plugin_id_t id, unsigned int cpu_index)
+{
+    CPUCount *count = g_ptr_array_index(counts, cpu_index);
+    g_autoptr(GString) report = g_string_new("");
+    gen_one_cpu_report(count, report);
+
+    if (report->len > 0) {
+        g_string_prepend(report, "Idling ");
+        qemu_plugin_outs(report->str);
+    }
 }
 
 static void vcpu_tb_exec(unsigned int cpu_index, void *udata)
 {
-    unsigned long n_insns = (unsigned long)udata;
+    CPUCount *count = max_cpus ?
+        g_ptr_array_index(counts, cpu_index) : &inline_count;
 
-    insn_count += n_insns;
-    bb_count++;
+    unsigned long n_insns = (unsigned long)udata;
+    g_mutex_lock(&count->lock);
+    count->insn_count += n_insns;
+    count->bb_count++;
+    g_mutex_unlock(&count->lock);
 }
 
 static void vcpu_tb_trans(qemu_plugin_id_t id, struct qemu_plugin_tb *tb)
@@ -42,9 +85,10 @@ static void vcpu_tb_trans(qemu_plugin_id_t id, struct qemu_plugin_tb *tb)
 
     if (do_inline) {
         qemu_plugin_register_vcpu_tb_exec_inline(tb, QEMU_PLUGIN_INLINE_ADD_U64,
-                                                 &bb_count, 1);
+                                                 &inline_count.bb_count, 1);
         qemu_plugin_register_vcpu_tb_exec_inline(tb, QEMU_PLUGIN_INLINE_ADD_U64,
-                                                 &insn_count, n_insns);
+                                                 &inline_count.insn_count,
+                                                 n_insns);
     } else {
         qemu_plugin_register_vcpu_tb_exec_cb(tb, vcpu_tb_exec,
                                              QEMU_PLUGIN_CB_NO_REGS,
@@ -56,8 +100,35 @@ QEMU_PLUGIN_EXPORT int qemu_plugin_install(qemu_plugin_id_t id,
                                            const qemu_info_t *info,
                                            int argc, char **argv)
 {
-    if (argc && strcmp(argv[0], "inline") == 0) {
-        do_inline = true;
+    int i;
+
+    for (i = 0; i < argc; i++) {
+        char *opt = argv[i];
+        if (g_strcmp0(opt, "inline") == 0) {
+            do_inline = true;
+        } else if (g_strcmp0(opt, "idle") == 0) {
+            idle_report = true;
+        } else {
+            fprintf(stderr, "option parsing failed: %s\n", opt);
+            return -1;
+        }
+    }
+
+    if (info->system_emulation && !do_inline) {
+        max_cpus = info->system.max_vcpus;
+        counts = g_ptr_array_new();
+        for (i = 0; i < max_cpus; i++) {
+            CPUCount *count = g_new0(CPUCount, 1);
+            g_mutex_init(&count->lock);
+            count->index = i;
+            g_ptr_array_add(counts, count);
+        }
+    } else if (!do_inline) {
+        g_mutex_init(&inline_count.lock);
+    }
+
+    if (idle_report) {
+        qemu_plugin_register_vcpu_idle_cb(id, vcpu_idle);
     }
 
     qemu_plugin_register_vcpu_tb_trans_cb(id, vcpu_tb_trans);
-- 
2.20.1

Re: [PULL 0/9] final misc fixes for 5.1-rc0

[Peter Maydell]

On Tue, 14 Jul 2020 at 10:52, Alex Bennée <alex.bennee@linaro.org> wrote:
>
> The following changes since commit 20c1df5476e1e9b5d3f5b94f9f3ce01d21f14c46:
>
>   Merge remote-tracking branch 'remotes/kraxel/tags/fixes-20200713-pull-request' into staging (2020-07-13 16:58:44 +0100)
>
> are available in the Git repository at:
>
>   https://github.com/stsquad/qemu.git tags/pull-misc-for-rc0-140720-1
>
> for you to fetch changes up to 4e6400c97024c2660b6db8aab1f0677744449f36:
>
>   plugins: expand the bb plugin to be thread safe and track per-cpu (2020-07-14 09:08:25 +0100)
>
> ----------------------------------------------------------------
> Final fixes for 5.1-rc0
>
>   - minor documentation nit
>   - clean all deps on re(configure)
>   - docker.py bootstrap fixes
>   - tweak containers.yml wildcards
>   - fix float16 nan detection
>   - conditional use of -Wpsabi
>   - fix missing iotlb data for plugins
>   - proper locking for helper based bb count

Fails "make check" on aarch32 and aarch64 hosts, because
eg qemu-system-arm segfaults on startup:

$ gdb --args ./arm-softmmu/qemu-system-arm -M virt -display none
[...]
(gdb) r
Starting program: /home/pm/qemu/build/all/arm-softmmu/qemu-system-arm
-M virt -display none
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/aarch64-linux-gnu/libthread_db.so.1".
[New Thread 0xffffe873e550 (LWP 1666261)]
[New Thread 0xffffe7f3d550 (LWP 1666262)]
[New Thread 0xffffe72e7550 (LWP 1666263)]

Thread 4 "qemu-system-arm" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0xffffe72e7550 (LWP 1666263)]
0x0000aaaaab1da004 in object_class_dynamic_cast_assert (class=0xaaaaabc209e0,
    typename=typename@entry=0xaaaaab33bfe8 "cpu",
file=file@entry=0xaaaaab3201a0 "/home/pm/qemu/accel/tcg/cputlb.c",
    line=line@entry=1025, func=func@entry=0xaaaaab3202f0
<__func__.35278> "tlb_fill") at /home/pm/qemu/qom/object.c:917
917         trace_object_class_dynamic_cast_assert(class ?
class->type->name : "(null)",
(gdb) bt
#0  0x0000aaaaab1da004 in object_class_dynamic_cast_assert
    (class=0xaaaaabc209e0, typename=typename@entry=0xaaaaab33bfe8
"cpu", file=file@entry=0xaaaaab3201a0
"/home/pm/qemu/accel/tcg/cputlb.c", line=line@entry=1025,
func=func@entry=0xaaaaab3202f0 <__func__.35278> "tlb_fill")
    at /home/pm/qemu/qom/object.c:917
#1  0x0000aaaaaaded754 in tlb_fill
    (cpu=0xaaaaabd301c0, addr=0, size=0, access_type=MMU_INST_FETCH,
mmu_idx=2, retaddr=0)
    at /home/pm/qemu/accel/tcg/cputlb.c:1025
#2  0x0000aaaaaadf0e08 in get_page_addr_code_hostp
(env=0xaaaaabd39870, addr=addr@entry=0, hostp=hostp@entry=0x0)
    at /home/pm/qemu/include/exec/cpu-all.h:451
#3  0x0000aaaaaadf0e70 in get_page_addr_code (env=<optimized out>,
addr=addr@entry=0)
    at /home/pm/qemu/accel/tcg/cputlb.c:1243
#4  0x0000aaaaaae08c30 in tb_htable_lookup
    (cpu=cpu@entry=0xaaaaabd301b0, pc=0, cs_base=<optimized out>,
flags=38928384, cf_mask=4278714368)
    at /home/pm/qemu/accel/tcg/cpu-exec.c:337
#5  0x0000aaaaaae09b14 in tb_lookup__cpu_state
    (cf_mask=<optimized out>, flags=0xffffe72e6b48,
cs_base=0xffffe72e6b40, pc=0xffffe72e6b44, cpu=0xaaaaabd301b0)
    at /home/pm/qemu/include/exec/tb-lookup.h:43
#6  tb_find (cf_mask=<optimized out>, tb_exit=0, last_tb=0x0,
cpu=0xaaaaabd301b0)
    at /home/pm/qemu/accel/tcg/cpu-exec.c:404
#7  cpu_exec (cpu=cpu@entry=0xaaaaabd301b0) at
/home/pm/qemu/accel/tcg/cpu-exec.c:731
#8  0x0000aaaaaaec45c4 in tcg_cpu_exec (cpu=0xaaaaabd301b0) at
/home/pm/qemu/softmmu/cpus.c:1356
#9  0x0000aaaaaaec69ec in qemu_tcg_cpu_thread_fn
(arg=arg@entry=0xaaaaabd301b0) at /home/pm/qemu/softmmu/cpus.c:1664
#10 0x0000aaaaab2e1758 in qemu_thread_start (args=<optimized out>) at
/home/pm/qemu/util/qemu-thread-posix.c:521
#11 0x0000fffff57be4fc in start_thread (arg=0xffffffffe10f) at
pthread_create.c:477
#12 0x0000fffff5716f2c in thread_start () at
../sysdeps/unix/sysv/linux/aarch64/clone.S:78

thanks
-- PMM

Re: [PULL 0/9] final misc fixes for 5.1-rc0

[Alex Bennée]

Peter Maydell <peter.maydell@linaro.org> writes:

> On Tue, 14 Jul 2020 at 10:52, Alex Bennée <alex.bennee@linaro.org> wrote:
>>
>> The following changes since commit 20c1df5476e1e9b5d3f5b94f9f3ce01d21f14c46:
>>
>>   Merge remote-tracking branch 'remotes/kraxel/tags/fixes-20200713-pull-request' into staging (2020-07-13 16:58:44 +0100)
>>
>> are available in the Git repository at:
>>
>>   https://github.com/stsquad/qemu.git tags/pull-misc-for-rc0-140720-1
>>
>> for you to fetch changes up to 4e6400c97024c2660b6db8aab1f0677744449f36:
>>
>>   plugins: expand the bb plugin to be thread safe and track per-cpu (2020-07-14 09:08:25 +0100)
>>
>> ----------------------------------------------------------------
>> Final fixes for 5.1-rc0
>>
>>   - minor documentation nit
>>   - clean all deps on re(configure)
>>   - docker.py bootstrap fixes
>>   - tweak containers.yml wildcards
>>   - fix float16 nan detection
>>   - conditional use of -Wpsabi
>>   - fix missing iotlb data for plugins
>>   - proper locking for helper based bb count
>
> Fails "make check" on aarch32 and aarch64 hosts, because
> eg qemu-system-arm segfaults on startup:
>
> $ gdb --args ./arm-softmmu/qemu-system-arm -M virt -display none
> [...]
> (gdb) r
> Starting program: /home/pm/qemu/build/all/arm-softmmu/qemu-system-arm
> -M virt -display none
> [Thread debugging using libthread_db enabled]
> Using host libthread_db library "/lib/aarch64-linux-gnu/libthread_db.so.1".
> [New Thread 0xffffe873e550 (LWP 1666261)]
> [New Thread 0xffffe7f3d550 (LWP 1666262)]
> [New Thread 0xffffe72e7550 (LWP 1666263)]
>
> Thread 4 "qemu-system-arm" received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 0xffffe72e7550 (LWP 1666263)]
> 0x0000aaaaab1da004 in object_class_dynamic_cast_assert (class=0xaaaaabc209e0,
>     typename=typename@entry=0xaaaaab33bfe8 "cpu",
> file=file@entry=0xaaaaab3201a0 "/home/pm/qemu/accel/tcg/cputlb.c",
>     line=line@entry=1025, func=func@entry=0xaaaaab3202f0
> <__func__.35278> "tlb_fill") at /home/pm/qemu/qom/object.c:917
> 917         trace_object_class_dynamic_cast_assert(class ?
> class->type->name : "(null)",
> (gdb) bt
> #0  0x0000aaaaab1da004 in object_class_dynamic_cast_assert
>     (class=0xaaaaabc209e0, typename=typename@entry=0xaaaaab33bfe8
> "cpu", file=file@entry=0xaaaaab3201a0
> "/home/pm/qemu/accel/tcg/cputlb.c", line=line@entry=1025,
> func=func@entry=0xaaaaab3202f0 <__func__.35278> "tlb_fill")
>     at /home/pm/qemu/qom/object.c:917
> #1  0x0000aaaaaaded754 in tlb_fill
>     (cpu=0xaaaaabd301c0, addr=0, size=0, access_type=MMU_INST_FETCH,
> mmu_idx=2, retaddr=0)
>     at /home/pm/qemu/accel/tcg/cputlb.c:1025
> #2  0x0000aaaaaadf0e08 in get_page_addr_code_hostp
> (env=0xaaaaabd39870, addr=addr@entry=0, hostp=hostp@entry=0x0)
>     at /home/pm/qemu/include/exec/cpu-all.h:451
> #3  0x0000aaaaaadf0e70 in get_page_addr_code (env=<optimized out>,
> addr=addr@entry=0)
>     at /home/pm/qemu/accel/tcg/cputlb.c:1243
> #4  0x0000aaaaaae08c30 in tb_htable_lookup
>     (cpu=cpu@entry=0xaaaaabd301b0, pc=0, cs_base=<optimized out>,
> flags=38928384, cf_mask=4278714368)
>     at /home/pm/qemu/accel/tcg/cpu-exec.c:337
> #5  0x0000aaaaaae09b14 in tb_lookup__cpu_state
>     (cf_mask=<optimized out>, flags=0xffffe72e6b48,
> cs_base=0xffffe72e6b40, pc=0xffffe72e6b44, cpu=0xaaaaabd301b0)
>     at /home/pm/qemu/include/exec/tb-lookup.h:43
> #6  tb_find (cf_mask=<optimized out>, tb_exit=0, last_tb=0x0,
> cpu=0xaaaaabd301b0)
>     at /home/pm/qemu/accel/tcg/cpu-exec.c:404
> #7  cpu_exec (cpu=cpu@entry=0xaaaaabd301b0) at
> /home/pm/qemu/accel/tcg/cpu-exec.c:731
> #8  0x0000aaaaaaec45c4 in tcg_cpu_exec (cpu=0xaaaaabd301b0) at
> /home/pm/qemu/softmmu/cpus.c:1356
> #9  0x0000aaaaaaec69ec in qemu_tcg_cpu_thread_fn
> (arg=arg@entry=0xaaaaabd301b0) at /home/pm/qemu/softmmu/cpus.c:1664
> #10 0x0000aaaaab2e1758 in qemu_thread_start (args=<optimized out>) at
> /home/pm/qemu/util/qemu-thread-posix.c:521
> #11 0x0000fffff57be4fc in start_thread (arg=0xffffffffe10f) at
> pthread_create.c:477
> #12 0x0000fffff5716f2c in thread_start () at
> ../sysdeps/unix/sysv/linux/aarch64/clone.S:78

This goes away after a make clean and rebuild. I suspect the commit:

  configure: remove all dependencies on a (re)configure

just causes the build to miss more of the bits on a rebuild. I'll drop
it for now and just live with the random syscall_nr.h files.

-- 
Alex Bennée

Re: [PULL 0/9] final misc fixes for 5.1-rc0

[Philippe Mathieu-Daudé]

Hi Peter,

On 7/15/20 10:06 AM, Peter Maydell wrote:
> On Tue, 14 Jul 2020 at 10:52, Alex Bennée <alex.bennee@linaro.org> wrote:
>>
>> The following changes since commit 20c1df5476e1e9b5d3f5b94f9f3ce01d21f14c46:
>>
>>   Merge remote-tracking branch 'remotes/kraxel/tags/fixes-20200713-pull-request' into staging (2020-07-13 16:58:44 +0100)
>>
>> are available in the Git repository at:
>>
>>   https://github.com/stsquad/qemu.git tags/pull-misc-for-rc0-140720-1
>>
>> for you to fetch changes up to 4e6400c97024c2660b6db8aab1f0677744449f36:
>>
>>   plugins: expand the bb plugin to be thread safe and track per-cpu (2020-07-14 09:08:25 +0100)
>>
>> ----------------------------------------------------------------
>> Final fixes for 5.1-rc0
>>
>>   - minor documentation nit
>>   - clean all deps on re(configure)
>>   - docker.py bootstrap fixes
>>   - tweak containers.yml wildcards
>>   - fix float16 nan detection
>>   - conditional use of -Wpsabi
>>   - fix missing iotlb data for plugins
>>   - proper locking for helper based bb count
> 
> Fails "make check" on aarch32 and aarch64 hosts, because
> eg qemu-system-arm segfaults on startup:

What host/distrib are you using?

I can not reproduce on aarch64 (Ubuntu 20.04 LTS) using:
gcc (Ubuntu 9.3.0-10ubuntu2) 9.3.0

Any config in particular?

> 
> $ gdb --args ./arm-softmmu/qemu-system-arm -M virt -display none
> [...]
> (gdb) r
> Starting program: /home/pm/qemu/build/all/arm-softmmu/qemu-system-arm
> -M virt -display none
> [Thread debugging using libthread_db enabled]
> Using host libthread_db library "/lib/aarch64-linux-gnu/libthread_db.so.1".
> [New Thread 0xffffe873e550 (LWP 1666261)]
> [New Thread 0xffffe7f3d550 (LWP 1666262)]
> [New Thread 0xffffe72e7550 (LWP 1666263)]
> 
> Thread 4 "qemu-system-arm" received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 0xffffe72e7550 (LWP 1666263)]
> 0x0000aaaaab1da004 in object_class_dynamic_cast_assert (class=0xaaaaabc209e0,
>     typename=typename@entry=0xaaaaab33bfe8 "cpu",
> file=file@entry=0xaaaaab3201a0 "/home/pm/qemu/accel/tcg/cputlb.c",
>     line=line@entry=1025, func=func@entry=0xaaaaab3202f0
> <__func__.35278> "tlb_fill") at /home/pm/qemu/qom/object.c:917
> 917         trace_object_class_dynamic_cast_assert(class ?
> class->type->name : "(null)",

Re: [PULL 0/9] final misc fixes for 5.1-rc0

[Peter Maydell]

On Wed, 15 Jul 2020 at 13:29, Philippe Mathieu-Daudé <f4bug@amsat.org> wrote:
> What host/distrib are you using?
>
> I can not reproduce on aarch64 (Ubuntu 20.04 LTS) using:
> gcc (Ubuntu 9.3.0-10ubuntu2) 9.3.0

Ubuntu 20.04 LTS (this is the aarch64.ci.qemu.org packet.net box
you have access to). Also Ubuntu 18.04.4 LTS aarch32 chroot on
an aarch64 system.

> Any config in particular?

Nope, stock "configure" with no arguments. Incremental build, not
from-clean.

thanks
-- PMM

Re: [PULL 0/9] final misc fixes for 5.1-rc0

[Philippe Mathieu-Daudé]

On 7/15/20 2:51 PM, Peter Maydell wrote:
> On Wed, 15 Jul 2020 at 13:29, Philippe Mathieu-Daudé <f4bug@amsat.org> wrote:
>> What host/distrib are you using?
>>
>> I can not reproduce on aarch64 (Ubuntu 20.04 LTS) using:
>> gcc (Ubuntu 9.3.0-10ubuntu2) 9.3.0
> 
> Ubuntu 20.04 LTS (this is the aarch64.ci.qemu.org packet.net box
> you have access to).

Yes this is where I tested.

> Also Ubuntu 18.04.4 LTS aarch32 chroot on
> an aarch64 system.
> 
>> Any config in particular?
> 
> Nope, stock "configure" with no arguments. Incremental build, not
> from-clean.

OK, I tested from-clean (in case that helps Alex).

Thanks,

Phil.