[PULL 00/16] various fixes for rc2 (semihosting, tcg, docker, tests/vm)

[PULL 00/16] various fixes for rc2 (semihosting, tcg, docker, tests/vm)

[Alex Bennée]

The following changes since commit 194f8ca825854abef3aceca1ed7eb5a53b08751f:

  Merge remote-tracking branch 'remotes/rth/tags/pull-tcg-20200725' into staging (2020-07-26 17:17:58 +0100)

are available in the Git repository at:

  https://github.com/stsquad/qemu.git tags/pull-fixes-for-rc2-270720-1

for you to fetch changes up to 4a70232b1d26b0d73e1bce60b2c3bdb7e4279d16:

  tests/vm: add shutdown timeout in basevm.py (2020-07-27 09:41:58 +0100)

----------------------------------------------------------------
Various fixes for rc2:

  - get shippable working again
  - semihosting bug fixes
  - tweak tb-size handling for low memory machines
  - i386 compound literal float fix
  - linux-user MAP_FIXED->MAP_NOREPLACE on fallback
  - docker binfmt_misc fixes
  - linux-user nanosleep fix
  - tests/vm drain console fixes

----------------------------------------------------------------
Alex Bennée (8):
      shippable: add one more qemu to registry url
      util: add qemu_get_host_physmem utility function
      util/oslib-win32: add qemu_get_host_physmem implementation
      accel/tcg: better handle memory constrained systems
      linux-user: don't use MAP_FIXED in pgd_find_hole_fallback
      tests/docker: fix update command due to python3 str/bytes distinction
      tests/docker: fix binfmt_misc image building
      tests/docker: add support for DEB_KEYRING

KONRAD Frederic (2):
      semihosting: defer connect_chardevs a little more to use serialx
      semihosting: don't send the trailing '\0'

Laszlo Ersek (1):
      target/i386: floatx80: avoid compound literals in static initializers

Laurent Vivier (2):
      linux-user: fix clock_nanosleep()
      linux-user, ppc: fix clock_nanosleep() for linux-user-ppc

Robert Foley (3):
      python/qemu: Cleanup changes to ConsoleSocket
      python/qemu: Change ConsoleSocket to optionally drain socket.
      tests/vm: add shutdown timeout in basevm.py

 include/fpu/softfloat.h                       |   1 +
 include/qemu/osdep.h                          |  15 +
 accel/tcg/translate-all.c                     |   7 +-
 hw/semihosting/console.c                      |   4 +-
 linux-user/elfload.c                          |  10 +-
 linux-user/syscall.c                          |  15 +-
 softmmu/vl.c                                  |   5 +-
 target/i386/fpu_helper.c                      | 426 +++++++++++++-------------
 util/oslib-posix.c                            |  15 +
 util/oslib-win32.c                            |  11 +
 .shippable.yml                                |   2 +-
 python/qemu/console_socket.py                 | 137 +++++----
 python/qemu/machine.py                        |  14 +-
 python/qemu/pylintrc                          |   2 +-
 tests/docker/Makefile.include                 |   2 +-
 tests/docker/docker.py                        |  13 +-
 tests/docker/dockerfiles/debian-bootstrap.pre |   7 +
 tests/vm/basevm.py                            |  15 +-
 18 files changed, 390 insertions(+), 311 deletions(-)

-- 
2.20.1

[PULL 01/16] shippable: add one more qemu to registry url

[Alex Bennée]

The registry url is <project>/<repo>/qemu/<image>

Perhaps we should rationalise that some day but for now.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20200724064509.331-2-alex.bennee@linaro.org>

diff --git a/.shippable.yml b/.shippable.yml
index f6b742432e5..89d8be4291b 100644
--- a/.shippable.yml
+++ b/.shippable.yml
@@ -27,7 +27,7 @@ env:
       TARGET_LIST=ppc64-softmmu,ppc64-linux-user,ppc64abi32-linux-user
 build:
   pre_ci_boot:
-    image_name: registry.gitlab.com/qemu-project/qemu/${IMAGE}
+    image_name: registry.gitlab.com/qemu-project/qemu/qemu/${IMAGE}
     image_tag: latest
     pull: true
     options: "-e HOME=/root"
-- 
2.20.1

[PULL 02/16] semihosting: defer connect_chardevs a little more to use serialx

[Alex Bennée]

From: KONRAD Frederic <frederic.konrad@adacore.com>

With that we can just use -semihosting-config chardev=serial0.

[AJB: tweak commit message]

Signed-off-by: KONRAD Frederic <frederic.konrad@adacore.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <1592215252-26742-1-git-send-email-frederic.konrad@adacore.com>
Message-Id: <20200724064509.331-3-alex.bennee@linaro.org>

diff --git a/softmmu/vl.c b/softmmu/vl.c
index 660537a7098..4eb9d1f7fd5 100644
--- a/softmmu/vl.c
+++ b/softmmu/vl.c
@@ -4127,8 +4127,6 @@ void qemu_init(int argc, char **argv, char **envp)
 
     qemu_opts_foreach(qemu_find_opts("chardev"),
                       chardev_init_func, NULL, &error_fatal);
-    /* now chardevs have been created we may have semihosting to connect */
-    qemu_semihosting_connect_chardevs();
 
 #ifdef CONFIG_VIRTFS
     qemu_opts_foreach(qemu_find_opts("fsdev"),
@@ -4279,6 +4277,9 @@ void qemu_init(int argc, char **argv, char **envp)
     if (foreach_device_config(DEV_DEBUGCON, debugcon_parse) < 0)
         exit(1);
 
+    /* now chardevs have been created we may have semihosting to connect */
+    qemu_semihosting_connect_chardevs();
+
     /* If no default VGA is requested, the default is "none".  */
     if (default_vga) {
         vga_model = get_default_vga_model(machine_class);
-- 
2.20.1

[PULL 03/16] semihosting: don't send the trailing '\0'

[Alex Bennée]

From: KONRAD Frederic <frederic.konrad@adacore.com>

Don't send the trailing 0 from the string.

Signed-off-by: KONRAD Frederic <frederic.konrad@adacore.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <1592215252-26742-2-git-send-email-frederic.konrad@adacore.com>
Message-Id: <20200724064509.331-4-alex.bennee@linaro.org>

diff --git a/hw/semihosting/console.c b/hw/semihosting/console.c
index 22e7827824a..9b4fee92602 100644
--- a/hw/semihosting/console.c
+++ b/hw/semihosting/console.c
@@ -52,7 +52,9 @@ static GString *copy_user_string(CPUArchState *env, target_ulong addr)
 
     do {
         if (cpu_memory_rw_debug(cpu, addr++, &c, 1, 0) == 0) {
-            s = g_string_append_c(s, c);
+            if (c) {
+                s = g_string_append_c(s, c);
+            }
         } else {
             qemu_log_mask(LOG_GUEST_ERROR,
                           "%s: passed inaccessible address " TARGET_FMT_lx,
-- 
2.20.1

[PULL 04/16] util: add qemu_get_host_physmem utility function

[Alex Bennée]

This will be used in a future patch. For POSIX systems _SC_PHYS_PAGES
isn't standardised but at least appears in the man pages for
Open/FreeBSD. The result is advisory so any users of it shouldn't just
fail if we can't work it out.

The win32 stub currently returns 0 until someone with a Windows system
can develop and test a patch.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Cc: BALATON Zoltan <balaton@eik.bme.hu>
Cc: Christian Ehrhardt <christian.ehrhardt@canonical.com>
Message-Id: <20200724064509.331-5-alex.bennee@linaro.org>

diff --git a/include/qemu/osdep.h b/include/qemu/osdep.h
index 45c217aa280..0b1298b3c91 100644
--- a/include/qemu/osdep.h
+++ b/include/qemu/osdep.h
@@ -668,4 +668,16 @@ static inline void qemu_reset_optind(void)
  */
 char *qemu_get_host_name(Error **errp);
 
+/**
+ * qemu_get_host_physmem:
+ *
+ * Operating system agnostic way of querying host memory.
+ *
+ * Returns amount of physical memory on the system. This is purely
+ * advisery and may return 0 if we can't work it out. At the other
+ * end we saturate to SIZE_MAX if you are lucky enough to have that
+ * much memory.
+ */
+size_t qemu_get_host_physmem(void);
+
 #endif
diff --git a/util/oslib-posix.c b/util/oslib-posix.c
index d9236746248..ad8001a4ad8 100644
--- a/util/oslib-posix.c
+++ b/util/oslib-posix.c
@@ -841,3 +841,18 @@ char *qemu_get_host_name(Error **errp)
 
     return g_steal_pointer(&hostname);
 }
+
+size_t qemu_get_host_physmem(void)
+{
+#ifdef _SC_PHYS_PAGES
+    long pages = sysconf(_SC_PHYS_PAGES);
+    if (pages > 0) {
+        if (pages > SIZE_MAX / qemu_real_host_page_size) {
+            return SIZE_MAX;
+        } else {
+            return pages * qemu_real_host_page_size;
+        }
+    }
+#endif
+    return 0;
+}
diff --git a/util/oslib-win32.c b/util/oslib-win32.c
index 7eedbe5859a..31030463cc9 100644
--- a/util/oslib-win32.c
+++ b/util/oslib-win32.c
@@ -828,3 +828,9 @@ char *qemu_get_host_name(Error **errp)
 
     return g_utf16_to_utf8(tmp, size, NULL, NULL, NULL);
 }
+
+size_t qemu_get_host_physmem(void)
+{
+    /* currently unimplemented */
+    return 0;
+}
-- 
2.20.1

[PULL 05/16] util/oslib-win32: add qemu_get_host_physmem implementation

[Alex Bennée]

Compile tested only.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Stefan Weil <sw@weilnetz.de>
Message-Id: <20200724064509.331-6-alex.bennee@linaro.org>

diff --git a/util/oslib-win32.c b/util/oslib-win32.c
index 31030463cc9..c654dafd937 100644
--- a/util/oslib-win32.c
+++ b/util/oslib-win32.c
@@ -831,6 +831,11 @@ char *qemu_get_host_name(Error **errp)
 
 size_t qemu_get_host_physmem(void)
 {
-    /* currently unimplemented */
+    MEMORYSTATUSEX statex;
+    statex.dwLength = sizeof(statex);
+
+    if (GlobalMemoryStatusEx(&statex)) {
+        return statex.ullTotalPhys;
+    }
     return 0;
 }
-- 
2.20.1

[PULL 06/16] accel/tcg: better handle memory constrained systems

[Alex Bennée]

It turns out there are some 64 bit systems that have relatively low
amounts of physical memory available to them (typically CI system).
Even with swapping available a 1GB translation buffer that fills up
can put the machine under increased memory pressure. Detect these low
memory situations and reduce tb_size appropriately.

Fixes: 600e17b2615 ("accel/tcg: increase default code gen buffer size for 64 bit")
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Robert Foley <robert.foley@linaro.org>
Cc: BALATON Zoltan <balaton@eik.bme.hu>
Cc: Christian Ehrhardt <christian.ehrhardt@canonical.com>
Message-Id: <20200724064509.331-7-alex.bennee@linaro.org>

diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c
index 2afa46bd2b1..2d83013633b 100644
--- a/accel/tcg/translate-all.c
+++ b/accel/tcg/translate-all.c
@@ -976,7 +976,12 @@ static inline size_t size_code_gen_buffer(size_t tb_size)
 {
     /* Size the buffer.  */
     if (tb_size == 0) {
-        tb_size = DEFAULT_CODE_GEN_BUFFER_SIZE;
+        size_t phys_mem = qemu_get_host_physmem();
+        if (phys_mem == 0) {
+            tb_size = DEFAULT_CODE_GEN_BUFFER_SIZE;
+        } else {
+            tb_size = MIN(DEFAULT_CODE_GEN_BUFFER_SIZE, phys_mem / 8);
+        }
     }
     if (tb_size < MIN_CODE_GEN_BUFFER_SIZE) {
         tb_size = MIN_CODE_GEN_BUFFER_SIZE;
-- 
2.20.1

Re: [PULL 06/16] accel/tcg: better handle memory constrained systems

[Christian Ehrhardt]

[-- Attachment #1: Type: text/plain, Size: 2677 bytes --]

On Mon, Jul 27, 2020 at 2:24 PM Alex Bennée <alex.bennee@linaro.org> wrote:

> It turns out there are some 64 bit systems that have relatively low
> amounts of physical memory available to them (typically CI system).
> Even with swapping available a 1GB translation buffer that fills up
> can put the machine under increased memory pressure. Detect these low
> memory situations and reduce tb_size appropriately.
>
> Fixes: 600e17b2615 ("accel/tcg: increase default code gen buffer size for
> 64 bit")
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
> Reviewed-by: Robert Foley <robert.foley@linaro.org>
> Cc: BALATON Zoltan <balaton@eik.bme.hu>
> Cc: Christian Ehrhardt <christian.ehrhardt@canonical.com>
> Message-Id: <20200724064509.331-7-alex.bennee@linaro.org>
>

I beg your pardon for the late reply, but I was out a week.
I see this is already the pull request and my former feedback was included
- thanks.

Never the less I took the chance to test it in the context that I found and
reported the initial bug.
If only to show that I didn't fire&forget this case :-)

We know there is quite some noise/deviation, but I only ran single tests as
the problem was easily visible despite the noise. Amount of memory qemu
settles on:

Host 32G, Guest 512M
4.2        633M
5.0       1672M
5.0+ Fix  1670M

Host 1.5G, Guest 512M
4.2        692M
5.0       16xxM (OOM)
5.0+ Fix   766M

So we seem to have achieved that small environments no more break (a very
small amount of very densely sized systems might still) but at the same
time get the bigger cache for any normal/large system.
Tested-by: Christian Ehrhardt <christian.ehrhardt@canonical.com>
Reviewed-by: Christian Ehrhardt <christian.ehrhardt@canonical.com>


> diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c
> index 2afa46bd2b1..2d83013633b 100644
> --- a/accel/tcg/translate-all.c
> +++ b/accel/tcg/translate-all.c
> @@ -976,7 +976,12 @@ static inline size_t size_code_gen_buffer(size_t
> tb_size)
>  {
>      /* Size the buffer.  */
>      if (tb_size == 0) {
> -        tb_size = DEFAULT_CODE_GEN_BUFFER_SIZE;
> +        size_t phys_mem = qemu_get_host_physmem();
> +        if (phys_mem == 0) {
> +            tb_size = DEFAULT_CODE_GEN_BUFFER_SIZE;
> +        } else {
> +            tb_size = MIN(DEFAULT_CODE_GEN_BUFFER_SIZE, phys_mem / 8);
> +        }
>      }
>      if (tb_size < MIN_CODE_GEN_BUFFER_SIZE) {
>          tb_size = MIN_CODE_GEN_BUFFER_SIZE;
> --
> 2.20.1
>
>

-- 
Christian Ehrhardt
Staff Engineer, Ubuntu Server
Canonical Ltd

[-- Attachment #2: Type: text/html, Size: 4050 bytes --]

[PULL 07/16] target/i386: floatx80: avoid compound literals in static initializers

[Alex Bennée]

From: Laszlo Ersek <lersek@redhat.com>

Quoting ISO C99 6.7.8p4, "All the expressions in an initializer for an
object that has static storage duration shall be constant expressions or
string literals".

The compound literal produced by the make_floatx80() macro is not such a
constant expression, per 6.6p7-9. (An implementation may accept it,
according to 6.6p10, but is not required to.)

Therefore using "floatx80_zero" and make_floatx80() for initializing
"f2xm1_table" and "fpatan_table" is not portable. And gcc-4.8 in RHEL-7.6
actually chokes on them:

> target/i386/fpu_helper.c:871:5: error: initializer element is not constant
>      { make_floatx80(0xbfff, 0x8000000000000000ULL),
>      ^

We've had the make_floatx80_init() macro for this purpose since commit
3bf7e40ab914 ("softfloat: fix for C99", 2012-03-17), so let's use that
macro again.

Fixes: eca30647fc0 ("target/i386: reimplement f2xm1 using floatx80 operations")
Fixes: ff57bb7b632 ("target/i386: reimplement fpatan using floatx80 operations")
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Cc: Aurelien Jarno <aurelien@aurel32.net>
Cc: Eduardo Habkost <ehabkost@redhat.com>
Cc: Joseph Myers <joseph@codesourcery.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Peter Maydell <peter.maydell@linaro.org>
Link: https://lists.gnu.org/archive/html/qemu-devel/2017-08/msg06566.html
Link: https://lists.gnu.org/archive/html/qemu-devel/2020-07/msg04714.html
Message-Id: <20200716144251.23004-1-lersek@redhat.com>
Message-Id: <20200724064509.331-8-alex.bennee@linaro.org>

diff --git a/include/fpu/softfloat.h b/include/fpu/softfloat.h
index f1a19df066b..659218b5c78 100644
--- a/include/fpu/softfloat.h
+++ b/include/fpu/softfloat.h
@@ -822,6 +822,7 @@ static inline bool floatx80_invalid_encoding(floatx80 a)
 }
 
 #define floatx80_zero make_floatx80(0x0000, 0x0000000000000000LL)
+#define floatx80_zero_init make_floatx80_init(0x0000, 0x0000000000000000LL)
 #define floatx80_one make_floatx80(0x3fff, 0x8000000000000000LL)
 #define floatx80_ln2 make_floatx80(0x3ffe, 0xb17217f7d1cf79acLL)
 #define floatx80_pi make_floatx80(0x4000, 0xc90fdaa22168c235LL)
diff --git a/target/i386/fpu_helper.c b/target/i386/fpu_helper.c
index f5e6c4b88d4..4ea73874d83 100644
--- a/target/i386/fpu_helper.c
+++ b/target/i386/fpu_helper.c
@@ -868,201 +868,201 @@ struct f2xm1_data {
 };
 
 static const struct f2xm1_data f2xm1_table[65] = {
-    { make_floatx80(0xbfff, 0x8000000000000000ULL),
-      make_floatx80(0x3ffe, 0x8000000000000000ULL),
-      make_floatx80(0xbffe, 0x8000000000000000ULL) },
-    { make_floatx80(0xbffe, 0xf800000000002e7eULL),
-      make_floatx80(0x3ffe, 0x82cd8698ac2b9160ULL),
-      make_floatx80(0xbffd, 0xfa64f2cea7a8dd40ULL) },
-    { make_floatx80(0xbffe, 0xefffffffffffe960ULL),
-      make_floatx80(0x3ffe, 0x85aac367cc488345ULL),
-      make_floatx80(0xbffd, 0xf4aa7930676ef976ULL) },
-    { make_floatx80(0xbffe, 0xe800000000006f10ULL),
-      make_floatx80(0x3ffe, 0x88980e8092da5c14ULL),
-      make_floatx80(0xbffd, 0xeecfe2feda4b47d8ULL) },
-    { make_floatx80(0xbffe, 0xe000000000008a45ULL),
-      make_floatx80(0x3ffe, 0x8b95c1e3ea8ba2a5ULL),
-      make_floatx80(0xbffd, 0xe8d47c382ae8bab6ULL) },
-    { make_floatx80(0xbffe, 0xd7ffffffffff8a9eULL),
-      make_floatx80(0x3ffe, 0x8ea4398b45cd8116ULL),
-      make_floatx80(0xbffd, 0xe2b78ce97464fdd4ULL) },
-    { make_floatx80(0xbffe, 0xd0000000000019a0ULL),
-      make_floatx80(0x3ffe, 0x91c3d373ab11b919ULL),
-      make_floatx80(0xbffd, 0xdc785918a9dc8dceULL) },
-    { make_floatx80(0xbffe, 0xc7ffffffffff14dfULL),
-      make_floatx80(0x3ffe, 0x94f4efa8fef76836ULL),
-      make_floatx80(0xbffd, 0xd61620ae02112f94ULL) },
-    { make_floatx80(0xbffe, 0xc000000000006530ULL),
-      make_floatx80(0x3ffe, 0x9837f0518db87fbbULL),
-      make_floatx80(0xbffd, 0xcf901f5ce48f008aULL) },
-    { make_floatx80(0xbffe, 0xb7ffffffffff1723ULL),
-      make_floatx80(0x3ffe, 0x9b8d39b9d54eb74cULL),
-      make_floatx80(0xbffd, 0xc8e58c8c55629168ULL) },
-    { make_floatx80(0xbffe, 0xb00000000000b5e1ULL),
-      make_floatx80(0x3ffe, 0x9ef5326091a0c366ULL),
-      make_floatx80(0xbffd, 0xc2159b3edcbe7934ULL) },
-    { make_floatx80(0xbffe, 0xa800000000006f8aULL),
-      make_floatx80(0x3ffe, 0xa27043030c49370aULL),
-      make_floatx80(0xbffd, 0xbb1f79f9e76d91ecULL) },
-    { make_floatx80(0xbffe, 0x9fffffffffff816aULL),
-      make_floatx80(0x3ffe, 0xa5fed6a9b15171cfULL),
-      make_floatx80(0xbffd, 0xb40252ac9d5d1c62ULL) },
-    { make_floatx80(0xbffe, 0x97ffffffffffb621ULL),
-      make_floatx80(0x3ffe, 0xa9a15ab4ea7c30e6ULL),
-      make_floatx80(0xbffd, 0xacbd4a962b079e34ULL) },
-    { make_floatx80(0xbffe, 0x8fffffffffff162bULL),
-      make_floatx80(0x3ffe, 0xad583eea42a1b886ULL),
-      make_floatx80(0xbffd, 0xa54f822b7abc8ef4ULL) },
-    { make_floatx80(0xbffe, 0x87ffffffffff4d34ULL),
-      make_floatx80(0x3ffe, 0xb123f581d2ac7b51ULL),
-      make_floatx80(0xbffd, 0x9db814fc5aa7095eULL) },
-    { make_floatx80(0xbffe, 0x800000000000227dULL),
-      make_floatx80(0x3ffe, 0xb504f333f9de539dULL),
-      make_floatx80(0xbffd, 0x95f619980c4358c6ULL) },
-    { make_floatx80(0xbffd, 0xefffffffffff3978ULL),
-      make_floatx80(0x3ffe, 0xb8fbaf4762fbd0a1ULL),
-      make_floatx80(0xbffd, 0x8e08a1713a085ebeULL) },
-    { make_floatx80(0xbffd, 0xe00000000000df81ULL),
-      make_floatx80(0x3ffe, 0xbd08a39f580bfd8cULL),
-      make_floatx80(0xbffd, 0x85eeb8c14fe804e8ULL) },
-    { make_floatx80(0xbffd, 0xd00000000000bccfULL),
-      make_floatx80(0x3ffe, 0xc12c4cca667062f6ULL),
-      make_floatx80(0xbffc, 0xfb4eccd6663e7428ULL) },
-    { make_floatx80(0xbffd, 0xc00000000000eff0ULL),
-      make_floatx80(0x3ffe, 0xc5672a1155069abeULL),
-      make_floatx80(0xbffc, 0xea6357baabe59508ULL) },
-    { make_floatx80(0xbffd, 0xb000000000000fe6ULL),
-      make_floatx80(0x3ffe, 0xc9b9bd866e2f234bULL),
-      make_floatx80(0xbffc, 0xd91909e6474372d4ULL) },
-    { make_floatx80(0xbffd, 0x9fffffffffff2172ULL),
-      make_floatx80(0x3ffe, 0xce248c151f84bf00ULL),
-      make_floatx80(0xbffc, 0xc76dcfab81ed0400ULL) },
-    { make_floatx80(0xbffd, 0x8fffffffffffafffULL),
-      make_floatx80(0x3ffe, 0xd2a81d91f12afb2bULL),
-      make_floatx80(0xbffc, 0xb55f89b83b541354ULL) },
-    { make_floatx80(0xbffc, 0xffffffffffff81a3ULL),
-      make_floatx80(0x3ffe, 0xd744fccad69d7d5eULL),
-      make_floatx80(0xbffc, 0xa2ec0cd4a58a0a88ULL) },
-    { make_floatx80(0xbffc, 0xdfffffffffff1568ULL),
-      make_floatx80(0x3ffe, 0xdbfbb797daf25a44ULL),
-      make_floatx80(0xbffc, 0x901121a0943696f0ULL) },
-    { make_floatx80(0xbffc, 0xbfffffffffff68daULL),
-      make_floatx80(0x3ffe, 0xe0ccdeec2a94f811ULL),
-      make_floatx80(0xbffb, 0xf999089eab583f78ULL) },
-    { make_floatx80(0xbffc, 0x9fffffffffff4690ULL),
-      make_floatx80(0x3ffe, 0xe5b906e77c83657eULL),
-      make_floatx80(0xbffb, 0xd237c8c41be4d410ULL) },
-    { make_floatx80(0xbffb, 0xffffffffffff8aeeULL),
-      make_floatx80(0x3ffe, 0xeac0c6e7dd24427cULL),
-      make_floatx80(0xbffb, 0xa9f9c8c116ddec20ULL) },
-    { make_floatx80(0xbffb, 0xbfffffffffff2d18ULL),
-      make_floatx80(0x3ffe, 0xefe4b99bdcdb06ebULL),
-      make_floatx80(0xbffb, 0x80da33211927c8a8ULL) },
-    { make_floatx80(0xbffa, 0xffffffffffff8ccbULL),
-      make_floatx80(0x3ffe, 0xf5257d152486d0f4ULL),
-      make_floatx80(0xbffa, 0xada82eadb792f0c0ULL) },
-    { make_floatx80(0xbff9, 0xffffffffffff11feULL),
-      make_floatx80(0x3ffe, 0xfa83b2db722a0846ULL),
-      make_floatx80(0xbff9, 0xaf89a491babef740ULL) },
-    { floatx80_zero,
-      make_floatx80(0x3fff, 0x8000000000000000ULL),
-      floatx80_zero },
-    { make_floatx80(0x3ff9, 0xffffffffffff2680ULL),
-      make_floatx80(0x3fff, 0x82cd8698ac2b9f6fULL),
-      make_floatx80(0x3ff9, 0xb361a62b0ae7dbc0ULL) },
-    { make_floatx80(0x3ffb, 0x800000000000b500ULL),
-      make_floatx80(0x3fff, 0x85aac367cc488345ULL),
-      make_floatx80(0x3ffa, 0xb5586cf9891068a0ULL) },
-    { make_floatx80(0x3ffb, 0xbfffffffffff4b67ULL),
-      make_floatx80(0x3fff, 0x88980e8092da7cceULL),
-      make_floatx80(0x3ffb, 0x8980e8092da7cce0ULL) },
-    { make_floatx80(0x3ffb, 0xffffffffffffff57ULL),
-      make_floatx80(0x3fff, 0x8b95c1e3ea8bd6dfULL),
-      make_floatx80(0x3ffb, 0xb95c1e3ea8bd6df0ULL) },
-    { make_floatx80(0x3ffc, 0x9fffffffffff811fULL),
-      make_floatx80(0x3fff, 0x8ea4398b45cd4780ULL),
-      make_floatx80(0x3ffb, 0xea4398b45cd47800ULL) },
-    { make_floatx80(0x3ffc, 0xbfffffffffff9980ULL),
-      make_floatx80(0x3fff, 0x91c3d373ab11b919ULL),
-      make_floatx80(0x3ffc, 0x8e1e9b9d588dc8c8ULL) },
-    { make_floatx80(0x3ffc, 0xdffffffffffff631ULL),
-      make_floatx80(0x3fff, 0x94f4efa8fef70864ULL),
-      make_floatx80(0x3ffc, 0xa7a77d47f7b84320ULL) },
-    { make_floatx80(0x3ffc, 0xffffffffffff2499ULL),
-      make_floatx80(0x3fff, 0x9837f0518db892d4ULL),
-      make_floatx80(0x3ffc, 0xc1bf828c6dc496a0ULL) },
-    { make_floatx80(0x3ffd, 0x8fffffffffff80fbULL),
-      make_floatx80(0x3fff, 0x9b8d39b9d54e3a79ULL),
-      make_floatx80(0x3ffc, 0xdc69cdceaa71d3c8ULL) },
-    { make_floatx80(0x3ffd, 0x9fffffffffffbc23ULL),
-      make_floatx80(0x3fff, 0x9ef5326091a10313ULL),
-      make_floatx80(0x3ffc, 0xf7a993048d081898ULL) },
-    { make_floatx80(0x3ffd, 0xafffffffffff20ecULL),
-      make_floatx80(0x3fff, 0xa27043030c49370aULL),
-      make_floatx80(0x3ffd, 0x89c10c0c3124dc28ULL) },
-    { make_floatx80(0x3ffd, 0xc00000000000fd2cULL),
-      make_floatx80(0x3fff, 0xa5fed6a9b15171cfULL),
-      make_floatx80(0x3ffd, 0x97fb5aa6c545c73cULL) },
-    { make_floatx80(0x3ffd, 0xd0000000000093beULL),
-      make_floatx80(0x3fff, 0xa9a15ab4ea7c30e6ULL),
-      make_floatx80(0x3ffd, 0xa6856ad3a9f0c398ULL) },
-    { make_floatx80(0x3ffd, 0xe00000000000c2aeULL),
-      make_floatx80(0x3fff, 0xad583eea42a17876ULL),
-      make_floatx80(0x3ffd, 0xb560fba90a85e1d8ULL) },
-    { make_floatx80(0x3ffd, 0xefffffffffff1e3fULL),
-      make_floatx80(0x3fff, 0xb123f581d2abef6cULL),
-      make_floatx80(0x3ffd, 0xc48fd6074aafbdb0ULL) },
-    { make_floatx80(0x3ffd, 0xffffffffffff1c23ULL),
-      make_floatx80(0x3fff, 0xb504f333f9de2cadULL),
-      make_floatx80(0x3ffd, 0xd413cccfe778b2b4ULL) },
-    { make_floatx80(0x3ffe, 0x8800000000006344ULL),
-      make_floatx80(0x3fff, 0xb8fbaf4762fbd0a1ULL),
-      make_floatx80(0x3ffd, 0xe3eebd1d8bef4284ULL) },
-    { make_floatx80(0x3ffe, 0x9000000000005d67ULL),
-      make_floatx80(0x3fff, 0xbd08a39f580c668dULL),
-      make_floatx80(0x3ffd, 0xf4228e7d60319a34ULL) },
-    { make_floatx80(0x3ffe, 0x9800000000009127ULL),
-      make_floatx80(0x3fff, 0xc12c4cca6670e042ULL),
-      make_floatx80(0x3ffe, 0x82589994cce1c084ULL) },
-    { make_floatx80(0x3ffe, 0x9fffffffffff06f9ULL),
-      make_floatx80(0x3fff, 0xc5672a11550655c3ULL),
-      make_floatx80(0x3ffe, 0x8ace5422aa0cab86ULL) },
-    { make_floatx80(0x3ffe, 0xa7fffffffffff80dULL),
-      make_floatx80(0x3fff, 0xc9b9bd866e2f234bULL),
-      make_floatx80(0x3ffe, 0x93737b0cdc5e4696ULL) },
-    { make_floatx80(0x3ffe, 0xafffffffffff1470ULL),
-      make_floatx80(0x3fff, 0xce248c151f83fd69ULL),
-      make_floatx80(0x3ffe, 0x9c49182a3f07fad2ULL) },
-    { make_floatx80(0x3ffe, 0xb800000000000e0aULL),
-      make_floatx80(0x3fff, 0xd2a81d91f12aec5cULL),
-      make_floatx80(0x3ffe, 0xa5503b23e255d8b8ULL) },
-    { make_floatx80(0x3ffe, 0xc00000000000b7faULL),
-      make_floatx80(0x3fff, 0xd744fccad69dd630ULL),
-      make_floatx80(0x3ffe, 0xae89f995ad3bac60ULL) },
-    { make_floatx80(0x3ffe, 0xc800000000003aa6ULL),
-      make_floatx80(0x3fff, 0xdbfbb797daf25a44ULL),
-      make_floatx80(0x3ffe, 0xb7f76f2fb5e4b488ULL) },
-    { make_floatx80(0x3ffe, 0xd00000000000a6aeULL),
-      make_floatx80(0x3fff, 0xe0ccdeec2a954685ULL),
-      make_floatx80(0x3ffe, 0xc199bdd8552a8d0aULL) },
-    { make_floatx80(0x3ffe, 0xd800000000004165ULL),
-      make_floatx80(0x3fff, 0xe5b906e77c837155ULL),
-      make_floatx80(0x3ffe, 0xcb720dcef906e2aaULL) },
-    { make_floatx80(0x3ffe, 0xe00000000000582cULL),
-      make_floatx80(0x3fff, 0xeac0c6e7dd24713aULL),
-      make_floatx80(0x3ffe, 0xd5818dcfba48e274ULL) },
-    { make_floatx80(0x3ffe, 0xe800000000001a5dULL),
-      make_floatx80(0x3fff, 0xefe4b99bdcdb06ebULL),
-      make_floatx80(0x3ffe, 0xdfc97337b9b60dd6ULL) },
-    { make_floatx80(0x3ffe, 0xefffffffffffc1efULL),
-      make_floatx80(0x3fff, 0xf5257d152486a2faULL),
-      make_floatx80(0x3ffe, 0xea4afa2a490d45f4ULL) },
-    { make_floatx80(0x3ffe, 0xf800000000001069ULL),
-      make_floatx80(0x3fff, 0xfa83b2db722a0e5cULL),
-      make_floatx80(0x3ffe, 0xf50765b6e4541cb8ULL) },
-    { make_floatx80(0x3fff, 0x8000000000000000ULL),
-      make_floatx80(0x4000, 0x8000000000000000ULL),
-      make_floatx80(0x3fff, 0x8000000000000000ULL) },
+    { make_floatx80_init(0xbfff, 0x8000000000000000ULL),
+      make_floatx80_init(0x3ffe, 0x8000000000000000ULL),
+      make_floatx80_init(0xbffe, 0x8000000000000000ULL) },
+    { make_floatx80_init(0xbffe, 0xf800000000002e7eULL),
+      make_floatx80_init(0x3ffe, 0x82cd8698ac2b9160ULL),
+      make_floatx80_init(0xbffd, 0xfa64f2cea7a8dd40ULL) },
+    { make_floatx80_init(0xbffe, 0xefffffffffffe960ULL),
+      make_floatx80_init(0x3ffe, 0x85aac367cc488345ULL),
+      make_floatx80_init(0xbffd, 0xf4aa7930676ef976ULL) },
+    { make_floatx80_init(0xbffe, 0xe800000000006f10ULL),
+      make_floatx80_init(0x3ffe, 0x88980e8092da5c14ULL),
+      make_floatx80_init(0xbffd, 0xeecfe2feda4b47d8ULL) },
+    { make_floatx80_init(0xbffe, 0xe000000000008a45ULL),
+      make_floatx80_init(0x3ffe, 0x8b95c1e3ea8ba2a5ULL),
+      make_floatx80_init(0xbffd, 0xe8d47c382ae8bab6ULL) },
+    { make_floatx80_init(0xbffe, 0xd7ffffffffff8a9eULL),
+      make_floatx80_init(0x3ffe, 0x8ea4398b45cd8116ULL),
+      make_floatx80_init(0xbffd, 0xe2b78ce97464fdd4ULL) },
+    { make_floatx80_init(0xbffe, 0xd0000000000019a0ULL),
+      make_floatx80_init(0x3ffe, 0x91c3d373ab11b919ULL),
+      make_floatx80_init(0xbffd, 0xdc785918a9dc8dceULL) },
+    { make_floatx80_init(0xbffe, 0xc7ffffffffff14dfULL),
+      make_floatx80_init(0x3ffe, 0x94f4efa8fef76836ULL),
+      make_floatx80_init(0xbffd, 0xd61620ae02112f94ULL) },
+    { make_floatx80_init(0xbffe, 0xc000000000006530ULL),
+      make_floatx80_init(0x3ffe, 0x9837f0518db87fbbULL),
+      make_floatx80_init(0xbffd, 0xcf901f5ce48f008aULL) },
+    { make_floatx80_init(0xbffe, 0xb7ffffffffff1723ULL),
+      make_floatx80_init(0x3ffe, 0x9b8d39b9d54eb74cULL),
+      make_floatx80_init(0xbffd, 0xc8e58c8c55629168ULL) },
+    { make_floatx80_init(0xbffe, 0xb00000000000b5e1ULL),
+      make_floatx80_init(0x3ffe, 0x9ef5326091a0c366ULL),
+      make_floatx80_init(0xbffd, 0xc2159b3edcbe7934ULL) },
+    { make_floatx80_init(0xbffe, 0xa800000000006f8aULL),
+      make_floatx80_init(0x3ffe, 0xa27043030c49370aULL),
+      make_floatx80_init(0xbffd, 0xbb1f79f9e76d91ecULL) },
+    { make_floatx80_init(0xbffe, 0x9fffffffffff816aULL),
+      make_floatx80_init(0x3ffe, 0xa5fed6a9b15171cfULL),
+      make_floatx80_init(0xbffd, 0xb40252ac9d5d1c62ULL) },
+    { make_floatx80_init(0xbffe, 0x97ffffffffffb621ULL),
+      make_floatx80_init(0x3ffe, 0xa9a15ab4ea7c30e6ULL),
+      make_floatx80_init(0xbffd, 0xacbd4a962b079e34ULL) },
+    { make_floatx80_init(0xbffe, 0x8fffffffffff162bULL),
+      make_floatx80_init(0x3ffe, 0xad583eea42a1b886ULL),
+      make_floatx80_init(0xbffd, 0xa54f822b7abc8ef4ULL) },
+    { make_floatx80_init(0xbffe, 0x87ffffffffff4d34ULL),
+      make_floatx80_init(0x3ffe, 0xb123f581d2ac7b51ULL),
+      make_floatx80_init(0xbffd, 0x9db814fc5aa7095eULL) },
+    { make_floatx80_init(0xbffe, 0x800000000000227dULL),
+      make_floatx80_init(0x3ffe, 0xb504f333f9de539dULL),
+      make_floatx80_init(0xbffd, 0x95f619980c4358c6ULL) },
+    { make_floatx80_init(0xbffd, 0xefffffffffff3978ULL),
+      make_floatx80_init(0x3ffe, 0xb8fbaf4762fbd0a1ULL),
+      make_floatx80_init(0xbffd, 0x8e08a1713a085ebeULL) },
+    { make_floatx80_init(0xbffd, 0xe00000000000df81ULL),
+      make_floatx80_init(0x3ffe, 0xbd08a39f580bfd8cULL),
+      make_floatx80_init(0xbffd, 0x85eeb8c14fe804e8ULL) },
+    { make_floatx80_init(0xbffd, 0xd00000000000bccfULL),
+      make_floatx80_init(0x3ffe, 0xc12c4cca667062f6ULL),
+      make_floatx80_init(0xbffc, 0xfb4eccd6663e7428ULL) },
+    { make_floatx80_init(0xbffd, 0xc00000000000eff0ULL),
+      make_floatx80_init(0x3ffe, 0xc5672a1155069abeULL),
+      make_floatx80_init(0xbffc, 0xea6357baabe59508ULL) },
+    { make_floatx80_init(0xbffd, 0xb000000000000fe6ULL),
+      make_floatx80_init(0x3ffe, 0xc9b9bd866e2f234bULL),
+      make_floatx80_init(0xbffc, 0xd91909e6474372d4ULL) },
+    { make_floatx80_init(0xbffd, 0x9fffffffffff2172ULL),
+      make_floatx80_init(0x3ffe, 0xce248c151f84bf00ULL),
+      make_floatx80_init(0xbffc, 0xc76dcfab81ed0400ULL) },
+    { make_floatx80_init(0xbffd, 0x8fffffffffffafffULL),
+      make_floatx80_init(0x3ffe, 0xd2a81d91f12afb2bULL),
+      make_floatx80_init(0xbffc, 0xb55f89b83b541354ULL) },
+    { make_floatx80_init(0xbffc, 0xffffffffffff81a3ULL),
+      make_floatx80_init(0x3ffe, 0xd744fccad69d7d5eULL),
+      make_floatx80_init(0xbffc, 0xa2ec0cd4a58a0a88ULL) },
+    { make_floatx80_init(0xbffc, 0xdfffffffffff1568ULL),
+      make_floatx80_init(0x3ffe, 0xdbfbb797daf25a44ULL),
+      make_floatx80_init(0xbffc, 0x901121a0943696f0ULL) },
+    { make_floatx80_init(0xbffc, 0xbfffffffffff68daULL),
+      make_floatx80_init(0x3ffe, 0xe0ccdeec2a94f811ULL),
+      make_floatx80_init(0xbffb, 0xf999089eab583f78ULL) },
+    { make_floatx80_init(0xbffc, 0x9fffffffffff4690ULL),
+      make_floatx80_init(0x3ffe, 0xe5b906e77c83657eULL),
+      make_floatx80_init(0xbffb, 0xd237c8c41be4d410ULL) },
+    { make_floatx80_init(0xbffb, 0xffffffffffff8aeeULL),
+      make_floatx80_init(0x3ffe, 0xeac0c6e7dd24427cULL),
+      make_floatx80_init(0xbffb, 0xa9f9c8c116ddec20ULL) },
+    { make_floatx80_init(0xbffb, 0xbfffffffffff2d18ULL),
+      make_floatx80_init(0x3ffe, 0xefe4b99bdcdb06ebULL),
+      make_floatx80_init(0xbffb, 0x80da33211927c8a8ULL) },
+    { make_floatx80_init(0xbffa, 0xffffffffffff8ccbULL),
+      make_floatx80_init(0x3ffe, 0xf5257d152486d0f4ULL),
+      make_floatx80_init(0xbffa, 0xada82eadb792f0c0ULL) },
+    { make_floatx80_init(0xbff9, 0xffffffffffff11feULL),
+      make_floatx80_init(0x3ffe, 0xfa83b2db722a0846ULL),
+      make_floatx80_init(0xbff9, 0xaf89a491babef740ULL) },
+    { floatx80_zero_init,
+      make_floatx80_init(0x3fff, 0x8000000000000000ULL),
+      floatx80_zero_init },
+    { make_floatx80_init(0x3ff9, 0xffffffffffff2680ULL),
+      make_floatx80_init(0x3fff, 0x82cd8698ac2b9f6fULL),
+      make_floatx80_init(0x3ff9, 0xb361a62b0ae7dbc0ULL) },
+    { make_floatx80_init(0x3ffb, 0x800000000000b500ULL),
+      make_floatx80_init(0x3fff, 0x85aac367cc488345ULL),
+      make_floatx80_init(0x3ffa, 0xb5586cf9891068a0ULL) },
+    { make_floatx80_init(0x3ffb, 0xbfffffffffff4b67ULL),
+      make_floatx80_init(0x3fff, 0x88980e8092da7cceULL),
+      make_floatx80_init(0x3ffb, 0x8980e8092da7cce0ULL) },
+    { make_floatx80_init(0x3ffb, 0xffffffffffffff57ULL),
+      make_floatx80_init(0x3fff, 0x8b95c1e3ea8bd6dfULL),
+      make_floatx80_init(0x3ffb, 0xb95c1e3ea8bd6df0ULL) },
+    { make_floatx80_init(0x3ffc, 0x9fffffffffff811fULL),
+      make_floatx80_init(0x3fff, 0x8ea4398b45cd4780ULL),
+      make_floatx80_init(0x3ffb, 0xea4398b45cd47800ULL) },
+    { make_floatx80_init(0x3ffc, 0xbfffffffffff9980ULL),
+      make_floatx80_init(0x3fff, 0x91c3d373ab11b919ULL),
+      make_floatx80_init(0x3ffc, 0x8e1e9b9d588dc8c8ULL) },
+    { make_floatx80_init(0x3ffc, 0xdffffffffffff631ULL),
+      make_floatx80_init(0x3fff, 0x94f4efa8fef70864ULL),
+      make_floatx80_init(0x3ffc, 0xa7a77d47f7b84320ULL) },
+    { make_floatx80_init(0x3ffc, 0xffffffffffff2499ULL),
+      make_floatx80_init(0x3fff, 0x9837f0518db892d4ULL),
+      make_floatx80_init(0x3ffc, 0xc1bf828c6dc496a0ULL) },
+    { make_floatx80_init(0x3ffd, 0x8fffffffffff80fbULL),
+      make_floatx80_init(0x3fff, 0x9b8d39b9d54e3a79ULL),
+      make_floatx80_init(0x3ffc, 0xdc69cdceaa71d3c8ULL) },
+    { make_floatx80_init(0x3ffd, 0x9fffffffffffbc23ULL),
+      make_floatx80_init(0x3fff, 0x9ef5326091a10313ULL),
+      make_floatx80_init(0x3ffc, 0xf7a993048d081898ULL) },
+    { make_floatx80_init(0x3ffd, 0xafffffffffff20ecULL),
+      make_floatx80_init(0x3fff, 0xa27043030c49370aULL),
+      make_floatx80_init(0x3ffd, 0x89c10c0c3124dc28ULL) },
+    { make_floatx80_init(0x3ffd, 0xc00000000000fd2cULL),
+      make_floatx80_init(0x3fff, 0xa5fed6a9b15171cfULL),
+      make_floatx80_init(0x3ffd, 0x97fb5aa6c545c73cULL) },
+    { make_floatx80_init(0x3ffd, 0xd0000000000093beULL),
+      make_floatx80_init(0x3fff, 0xa9a15ab4ea7c30e6ULL),
+      make_floatx80_init(0x3ffd, 0xa6856ad3a9f0c398ULL) },
+    { make_floatx80_init(0x3ffd, 0xe00000000000c2aeULL),
+      make_floatx80_init(0x3fff, 0xad583eea42a17876ULL),
+      make_floatx80_init(0x3ffd, 0xb560fba90a85e1d8ULL) },
+    { make_floatx80_init(0x3ffd, 0xefffffffffff1e3fULL),
+      make_floatx80_init(0x3fff, 0xb123f581d2abef6cULL),
+      make_floatx80_init(0x3ffd, 0xc48fd6074aafbdb0ULL) },
+    { make_floatx80_init(0x3ffd, 0xffffffffffff1c23ULL),
+      make_floatx80_init(0x3fff, 0xb504f333f9de2cadULL),
+      make_floatx80_init(0x3ffd, 0xd413cccfe778b2b4ULL) },
+    { make_floatx80_init(0x3ffe, 0x8800000000006344ULL),
+      make_floatx80_init(0x3fff, 0xb8fbaf4762fbd0a1ULL),
+      make_floatx80_init(0x3ffd, 0xe3eebd1d8bef4284ULL) },
+    { make_floatx80_init(0x3ffe, 0x9000000000005d67ULL),
+      make_floatx80_init(0x3fff, 0xbd08a39f580c668dULL),
+      make_floatx80_init(0x3ffd, 0xf4228e7d60319a34ULL) },
+    { make_floatx80_init(0x3ffe, 0x9800000000009127ULL),
+      make_floatx80_init(0x3fff, 0xc12c4cca6670e042ULL),
+      make_floatx80_init(0x3ffe, 0x82589994cce1c084ULL) },
+    { make_floatx80_init(0x3ffe, 0x9fffffffffff06f9ULL),
+      make_floatx80_init(0x3fff, 0xc5672a11550655c3ULL),
+      make_floatx80_init(0x3ffe, 0x8ace5422aa0cab86ULL) },
+    { make_floatx80_init(0x3ffe, 0xa7fffffffffff80dULL),
+      make_floatx80_init(0x3fff, 0xc9b9bd866e2f234bULL),
+      make_floatx80_init(0x3ffe, 0x93737b0cdc5e4696ULL) },
+    { make_floatx80_init(0x3ffe, 0xafffffffffff1470ULL),
+      make_floatx80_init(0x3fff, 0xce248c151f83fd69ULL),
+      make_floatx80_init(0x3ffe, 0x9c49182a3f07fad2ULL) },
+    { make_floatx80_init(0x3ffe, 0xb800000000000e0aULL),
+      make_floatx80_init(0x3fff, 0xd2a81d91f12aec5cULL),
+      make_floatx80_init(0x3ffe, 0xa5503b23e255d8b8ULL) },
+    { make_floatx80_init(0x3ffe, 0xc00000000000b7faULL),
+      make_floatx80_init(0x3fff, 0xd744fccad69dd630ULL),
+      make_floatx80_init(0x3ffe, 0xae89f995ad3bac60ULL) },
+    { make_floatx80_init(0x3ffe, 0xc800000000003aa6ULL),
+      make_floatx80_init(0x3fff, 0xdbfbb797daf25a44ULL),
+      make_floatx80_init(0x3ffe, 0xb7f76f2fb5e4b488ULL) },
+    { make_floatx80_init(0x3ffe, 0xd00000000000a6aeULL),
+      make_floatx80_init(0x3fff, 0xe0ccdeec2a954685ULL),
+      make_floatx80_init(0x3ffe, 0xc199bdd8552a8d0aULL) },
+    { make_floatx80_init(0x3ffe, 0xd800000000004165ULL),
+      make_floatx80_init(0x3fff, 0xe5b906e77c837155ULL),
+      make_floatx80_init(0x3ffe, 0xcb720dcef906e2aaULL) },
+    { make_floatx80_init(0x3ffe, 0xe00000000000582cULL),
+      make_floatx80_init(0x3fff, 0xeac0c6e7dd24713aULL),
+      make_floatx80_init(0x3ffe, 0xd5818dcfba48e274ULL) },
+    { make_floatx80_init(0x3ffe, 0xe800000000001a5dULL),
+      make_floatx80_init(0x3fff, 0xefe4b99bdcdb06ebULL),
+      make_floatx80_init(0x3ffe, 0xdfc97337b9b60dd6ULL) },
+    { make_floatx80_init(0x3ffe, 0xefffffffffffc1efULL),
+      make_floatx80_init(0x3fff, 0xf5257d152486a2faULL),
+      make_floatx80_init(0x3ffe, 0xea4afa2a490d45f4ULL) },
+    { make_floatx80_init(0x3ffe, 0xf800000000001069ULL),
+      make_floatx80_init(0x3fff, 0xfa83b2db722a0e5cULL),
+      make_floatx80_init(0x3ffe, 0xf50765b6e4541cb8ULL) },
+    { make_floatx80_init(0x3fff, 0x8000000000000000ULL),
+      make_floatx80_init(0x4000, 0x8000000000000000ULL),
+      make_floatx80_init(0x3fff, 0x8000000000000000ULL) },
 };
 
 void helper_f2xm1(CPUX86State *env)
@@ -1275,24 +1275,24 @@ struct fpatan_data {
 };
 
 static const struct fpatan_data fpatan_table[9] = {
-    { floatx80_zero,
-      floatx80_zero },
-    { make_floatx80(0x3ffb, 0xfeadd4d5617b6e33ULL),
-      make_floatx80(0xbfb9, 0xdda19d8305ddc420ULL) },
-    { make_floatx80(0x3ffc, 0xfadbafc96406eb15ULL),
-      make_floatx80(0x3fbb, 0xdb8f3debef442fccULL) },
-    { make_floatx80(0x3ffd, 0xb7b0ca0f26f78474ULL),
-      make_floatx80(0xbfbc, 0xeab9bdba460376faULL) },
-    { make_floatx80(0x3ffd, 0xed63382b0dda7b45ULL),
-      make_floatx80(0x3fbc, 0xdfc88bd978751a06ULL) },
-    { make_floatx80(0x3ffe, 0x8f005d5ef7f59f9bULL),
-      make_floatx80(0x3fbd, 0xb906bc2ccb886e90ULL) },
-    { make_floatx80(0x3ffe, 0xa4bc7d1934f70924ULL),
-      make_floatx80(0x3fbb, 0xcd43f9522bed64f8ULL) },
-    { make_floatx80(0x3ffe, 0xb8053e2bc2319e74ULL),
-      make_floatx80(0xbfbc, 0xd3496ab7bd6eef0cULL) },
-    { make_floatx80(0x3ffe, 0xc90fdaa22168c235ULL),
-      make_floatx80(0xbfbc, 0xece675d1fc8f8cbcULL) },
+    { floatx80_zero_init,
+      floatx80_zero_init },
+    { make_floatx80_init(0x3ffb, 0xfeadd4d5617b6e33ULL),
+      make_floatx80_init(0xbfb9, 0xdda19d8305ddc420ULL) },
+    { make_floatx80_init(0x3ffc, 0xfadbafc96406eb15ULL),
+      make_floatx80_init(0x3fbb, 0xdb8f3debef442fccULL) },
+    { make_floatx80_init(0x3ffd, 0xb7b0ca0f26f78474ULL),
+      make_floatx80_init(0xbfbc, 0xeab9bdba460376faULL) },
+    { make_floatx80_init(0x3ffd, 0xed63382b0dda7b45ULL),
+      make_floatx80_init(0x3fbc, 0xdfc88bd978751a06ULL) },
+    { make_floatx80_init(0x3ffe, 0x8f005d5ef7f59f9bULL),
+      make_floatx80_init(0x3fbd, 0xb906bc2ccb886e90ULL) },
+    { make_floatx80_init(0x3ffe, 0xa4bc7d1934f70924ULL),
+      make_floatx80_init(0x3fbb, 0xcd43f9522bed64f8ULL) },
+    { make_floatx80_init(0x3ffe, 0xb8053e2bc2319e74ULL),
+      make_floatx80_init(0xbfbc, 0xd3496ab7bd6eef0cULL) },
+    { make_floatx80_init(0x3ffe, 0xc90fdaa22168c235ULL),
+      make_floatx80_init(0xbfbc, 0xece675d1fc8f8cbcULL) },
 };
 
 void helper_fpatan(CPUX86State *env)
-- 
2.20.1

[PULL 08/16] linux-user: don't use MAP_FIXED in pgd_find_hole_fallback

[Alex Bennée]

Plain MAP_FIXED has the undesirable behaviour of splatting exiting
maps so we don't actually achieve what we want when looking for gaps.
We should be using MAP_FIXED_NOREPLACE. As this isn't always available
we need to potentially check the returned address to see if the kernel
gave us what we asked for.

Fixes: ad592e37dfc ("linux-user: provide fallback pgd_find_hole for bare chroots")
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20200724064509.331-9-alex.bennee@linaro.org>

diff --git a/include/qemu/osdep.h b/include/qemu/osdep.h
index 0b1298b3c91..20872e793e4 100644
--- a/include/qemu/osdep.h
+++ b/include/qemu/osdep.h
@@ -173,6 +173,9 @@ extern int daemon(int, int);
 #ifndef MAP_ANONYMOUS
 #define MAP_ANONYMOUS MAP_ANON
 #endif
+#ifndef MAP_FIXED_NOREPLACE
+#define MAP_FIXED_NOREPLACE 0
+#endif
 #ifndef ENOMEDIUM
 #define ENOMEDIUM ENODEV
 #endif
diff --git a/linux-user/elfload.c b/linux-user/elfload.c
index 7e7f642332d..fe9dfe795dd 100644
--- a/linux-user/elfload.c
+++ b/linux-user/elfload.c
@@ -2134,12 +2134,15 @@ static uintptr_t pgd_find_hole_fallback(uintptr_t guest_size, uintptr_t brk,
             /* we have run out of space */
             return -1;
         } else {
-            int flags = MAP_ANONYMOUS | MAP_PRIVATE | MAP_NORESERVE | MAP_FIXED;
+            int flags = MAP_ANONYMOUS | MAP_PRIVATE | MAP_NORESERVE |
+                MAP_FIXED_NOREPLACE;
             void * mmap_start = mmap((void *) align_start, guest_size,
                                      PROT_NONE, flags, -1, 0);
             if (mmap_start != MAP_FAILED) {
                 munmap((void *) align_start, guest_size);
-                return (uintptr_t) mmap_start + offset;
+                if (MAP_FIXED_NOREPLACE || mmap_start == (void *) align_start) {
+                    return (uintptr_t) mmap_start + offset;
+                }
             }
             base += qemu_host_page_size;
         }
@@ -2307,9 +2310,8 @@ static void pgb_reserved_va(const char *image_name, abi_ulong guest_loaddr,
     /* Widen the "image" to the entire reserved address space. */
     pgb_static(image_name, 0, reserved_va, align);
 
-#ifdef MAP_FIXED_NOREPLACE
+    /* osdep.h defines this as 0 if it's missing */
     flags |= MAP_FIXED_NOREPLACE;
-#endif
 
     /* Reserve the memory on the host. */
     assert(guest_base != 0);
-- 
2.20.1

Re: [PULL 08/16] linux-user: don't use MAP_FIXED in pgd_find_hole_fallback

[Peter Maydell]

On Mon, 27 Jul 2020 at 13:24, Alex Bennée <alex.bennee@linaro.org> wrote:
>
> Plain MAP_FIXED has the undesirable behaviour of splatting exiting
> maps so we don't actually achieve what we want when looking for gaps.
> We should be using MAP_FIXED_NOREPLACE. As this isn't always available
> we need to potentially check the returned address to see if the kernel
> gave us what we asked for.
>
> Fixes: ad592e37dfc ("linux-user: provide fallback pgd_find_hole for bare chroots")
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
> Message-Id: <20200724064509.331-9-alex.bennee@linaro.org>

Hi; Coverity thinks this conditional expression is suspicious
(CID 1431059):

>              if (mmap_start != MAP_FAILED) {
>                  munmap((void *) align_start, guest_size);
> -                return (uintptr_t) mmap_start + offset;
> +                if (MAP_FIXED_NOREPLACE || mmap_start == (void *) align_start) {

because it's performing a logical OR operation where the left
operand is an integer constant that's neither 0 nor 1
(it's 1048576). What was this intended to be?

> +                    return (uintptr_t) mmap_start + offset;
> +                }
>              }

thanks
-- PMM

Re: [PULL 08/16] linux-user: don't use MAP_FIXED in pgd_find_hole_fallback

[Alex Bennée]

Peter Maydell <peter.maydell@linaro.org> writes:

> On Mon, 27 Jul 2020 at 13:24, Alex Bennée <alex.bennee@linaro.org> wrote:
>>
>> Plain MAP_FIXED has the undesirable behaviour of splatting exiting
>> maps so we don't actually achieve what we want when looking for gaps.
>> We should be using MAP_FIXED_NOREPLACE. As this isn't always available
>> we need to potentially check the returned address to see if the kernel
>> gave us what we asked for.
>>
>> Fixes: ad592e37dfc ("linux-user: provide fallback pgd_find_hole for bare chroots")
>> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
>> Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
>> Message-Id: <20200724064509.331-9-alex.bennee@linaro.org>
>
> Hi; Coverity thinks this conditional expression is suspicious
> (CID 1431059):
>
>>              if (mmap_start != MAP_FAILED) {
>>                  munmap((void *) align_start, guest_size);
>> -                return (uintptr_t) mmap_start + offset;
>> +                if (MAP_FIXED_NOREPLACE || mmap_start == (void *) align_start) {
>
> because it's performing a logical OR operation where the left
> operand is an integer constant that's neither 0 nor 1
> (it's 1048576). What was this intended to be?

It's 0 if the header doesn't provide it. If it's !0 we don't need to
check the address because it should have been in the correct place.

>
>> +                    return (uintptr_t) mmap_start + offset;
>> +                }
>>              }
>
> thanks
> -- PMM


-- 
Alex Bennée

Re: [PULL 08/16] linux-user: don't use MAP_FIXED in pgd_find_hole_fallback

[Peter Maydell]

On Tue, 28 Jul 2020 at 17:04, Alex Bennée <alex.bennee@linaro.org> wrote:
> Peter Maydell <peter.maydell@linaro.org> writes:
> > Hi; Coverity thinks this conditional expression is suspicious
> > (CID 1431059):
> >
> >>              if (mmap_start != MAP_FAILED) {
> >>                  munmap((void *) align_start, guest_size);
> >> -                return (uintptr_t) mmap_start + offset;
> >> +                if (MAP_FIXED_NOREPLACE || mmap_start == (void *) align_start) {
> >
> > because it's performing a logical OR operation where the left
> > operand is an integer constant that's neither 0 nor 1
> > (it's 1048576). What was this intended to be?
>
> It's 0 if the header doesn't provide it. If it's !0 we don't need to
> check the address because it should have been in the correct place.

OK. "if (MAP_FIXED_NOREPLACE != 0 || ...)" will probably satisfy
Coverity then.

-- PMM

[PULL 09/16] tests/docker: fix update command due to python3 str/bytes distinction

[Alex Bennée]

Does this seem convoluted to you? It feels a little complicated to me.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200724064509.331-10-alex.bennee@linaro.org>

diff --git a/tests/docker/docker.py b/tests/docker/docker.py
index c9f20d8d093..356d7618f1a 100755
--- a/tests/docker/docker.py
+++ b/tests/docker/docker.py
@@ -24,7 +24,7 @@ import tempfile
 import re
 import signal
 from tarfile import TarFile, TarInfo
-from io import StringIO
+from io import StringIO, BytesIO
 from shutil import copy, rmtree
 from pwd import getpwuid
 from datetime import datetime, timedelta
@@ -541,13 +541,14 @@ class UpdateCommand(SubCommand):
 
         # Create a Docker buildfile
         df = StringIO()
-        df.write("FROM %s\n" % args.tag)
-        df.write("ADD . /\n")
-        df.seek(0)
+        df.write(u"FROM %s\n" % args.tag)
+        df.write(u"ADD . /\n")
+
+        df_bytes = BytesIO(bytes(df.getvalue(), "UTF-8"))
 
         df_tar = TarInfo(name="Dockerfile")
-        df_tar.size = len(df.buf)
-        tmp_tar.addfile(df_tar, fileobj=df)
+        df_tar.size = df_bytes.getbuffer().nbytes
+        tmp_tar.addfile(df_tar, fileobj=df_bytes)
 
         tmp_tar.close()
 
-- 
2.20.1

[PULL 10/16] tests/docker: fix binfmt_misc image building

[Alex Bennée]

When we updated the arguments for docker.py we missed a bit.

Fixes: dfae628459 ("docker.py/build: support -t and -f arguments")
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20200724064509.331-11-alex.bennee@linaro.org>

diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index a104e9df281..9119dff97de 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -78,7 +78,7 @@ docker-binfmt-image-debian-%: $(DOCKER_FILES_DIR)/debian-bootstrap.docker
 			DEB_ARCH=$(DEB_ARCH)					\
 			DEB_TYPE=$(DEB_TYPE) 					\
 			$(if $(DEB_URL),DEB_URL=$(DEB_URL),)			\
-			$(DOCKER_SCRIPT) build qemu/debian-$* $< 		\
+			$(DOCKER_SCRIPT) build -t qemu/debian-$* -f $< 		\
 			$(if $V,,--quiet) $(if $(NOCACHE),--no-cache) 		\
 			$(if $(NOUSER),,--add-current-user) 			\
 			$(if $(EXTRA_FILES),--extra-files $(EXTRA_FILES))	\
-- 
2.20.1

[PULL 11/16] tests/docker: add support for DEB_KEYRING

[Alex Bennée]

For installing stuff from sid or ports you may need to manually
specify the location of the keyring. You can even import keys into
your personal keyring and point it there, e.g.:

  gpg --keyserver keyring.debian.org --recv-keys 84C573CD4E1AFD6C
  make docker-binfmt-image-debian-sid-hppa DEB_TYPE=sid DEB_ARCH=hppa \
      DEB_URL=http://ftp.ports.debian.org/debian-ports/ \
      EXECUTABLE=./hppa-linux-user/qemu-hppa V=1 \
      DEB_KEYRING=${HOME}/.gnupg/pubring.kbx

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20200724064509.331-12-alex.bennee@linaro.org>

diff --git a/tests/docker/dockerfiles/debian-bootstrap.pre b/tests/docker/dockerfiles/debian-bootstrap.pre
index c164778c302..35c85f7db8a 100755
--- a/tests/docker/dockerfiles/debian-bootstrap.pre
+++ b/tests/docker/dockerfiles/debian-bootstrap.pre
@@ -79,6 +79,13 @@ else
     fi
 fi
 
+#
+# Add optional args
+#
+if [ -n "${DEB_KEYRING}" ]; then
+    DEBOOTSTRAP="${DEBOOTSTRAP} --keyring=${DEB_KEYRING}"
+fi
+
 #
 # Finally check to see if any qemu's are installed
 #
-- 
2.20.1

[PULL 12/16] linux-user: fix clock_nanosleep()

[Alex Bennée]

From: Laurent Vivier <laurent@vivier.eu>

If the call is interrupted by a signal handler, it fails with error EINTR
and if "remain" is not NULL and "flags" is not TIMER_ABSTIME, it returns
the remaining unslept time in "remain".

Update linux-user to not overwrite the "remain" structure if there is no
error.

Found with "make check-tcg", linux-test fails on nanosleep test:

  TEST    linux-test on x86_64
.../tests/tcg/multiarch/linux-test.c:242: nanosleep

Reported-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20200722174612.2917566-2-laurent@vivier.eu>
Message-Id: <20200724064509.331-13-alex.bennee@linaro.org>

diff --git a/linux-user/syscall.c b/linux-user/syscall.c
index 1211e759c26..43a6e283961 100644
--- a/linux-user/syscall.c
+++ b/linux-user/syscall.c
@@ -11831,8 +11831,14 @@ static abi_long do_syscall1(void *cpu_env, int num, abi_long arg1,
         target_to_host_timespec(&ts, arg3);
         ret = get_errno(safe_clock_nanosleep(arg1, arg2,
                                              &ts, arg4 ? &ts : NULL));
-        if (arg4)
+        /*
+         * if the call is interrupted by a signal handler, it fails
+         * with error -TARGET_EINTR and if arg4 is not NULL and arg2 is not
+         * TIMER_ABSTIME, it returns the remaining unslept time in arg4.
+         */
+        if (ret == -TARGET_EINTR && arg4 && arg2 != TIMER_ABSTIME) {
             host_to_target_timespec(arg4, &ts);
+        }
 
 #if defined(TARGET_PPC)
         /* clock_nanosleep is odd in that it returns positive errno values.
-- 
2.20.1

[PULL 13/16] linux-user, ppc: fix clock_nanosleep() for linux-user-ppc

[Alex Bennée]

From: Laurent Vivier <laurent@vivier.eu>

Our safe_clock_nanosleep() returns -1 and updates errno.

We don't need to update the CRF bit in syscall.c because it will
be updated in ppc/cpu_loop.c as the return value is negative.

Signed-off-by: Laurent Vivier <laurent@vivier.eu>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20200722174612.2917566-3-laurent@vivier.eu>
Message-Id: <20200724064509.331-14-alex.bennee@linaro.org>

diff --git a/linux-user/syscall.c b/linux-user/syscall.c
index 43a6e283961..f5c4f6b95db 100644
--- a/linux-user/syscall.c
+++ b/linux-user/syscall.c
@@ -11840,13 +11840,6 @@ static abi_long do_syscall1(void *cpu_env, int num, abi_long arg1,
             host_to_target_timespec(arg4, &ts);
         }
 
-#if defined(TARGET_PPC)
-        /* clock_nanosleep is odd in that it returns positive errno values.
-         * On PPC, CR0 bit 3 should be set in such a situation. */
-        if (ret && ret != -TARGET_ERESTARTSYS) {
-            ((CPUPPCState *)cpu_env)->crf[0] |= 1;
-        }
-#endif
         return ret;
     }
 #endif
-- 
2.20.1

[PULL 14/16] python/qemu: Cleanup changes to ConsoleSocket

[Alex Bennée]

From: Robert Foley <robert.foley@linaro.org>

The changes to console_socket.py and machine.py are to
cleanup for pylint and flake8.

Signed-off-by: Robert Foley <robert.foley@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20200717203041.9867-2-robert.foley@linaro.org>
Message-Id: <20200724064509.331-15-alex.bennee@linaro.org>

diff --git a/python/qemu/console_socket.py b/python/qemu/console_socket.py
index 830cb7c6282..09986bc2152 100644
--- a/python/qemu/console_socket.py
+++ b/python/qemu/console_socket.py
@@ -1,12 +1,9 @@
-#!/usr/bin/env python3
-#
-# This python module implements a ConsoleSocket object which is
-# designed always drain the socket itself, and place
-# the bytes into a in memory buffer for later processing.
-#
-# Optionally a file path can be passed in and we will also
-# dump the characters to this file for debug.
-#
+"""
+QEMU Console Socket Module:
+
+This python module implements a ConsoleSocket object,
+which can drain a socket and optionally dump the bytes to file.
+"""
 # Copyright 2020 Linaro
 #
 # Authors:
@@ -15,20 +12,27 @@
 # This code is licensed under the GPL version 2 or later.  See
 # the COPYING file in the top-level directory.
 #
+
 import asyncore
 import socket
 import threading
-import io
-import os
-import sys
 from collections import deque
 import time
-import traceback
+
 
 class ConsoleSocket(asyncore.dispatcher):
+    """
+    ConsoleSocket represents a socket attached to a char device.
 
+    Drains the socket and places the bytes into an in memory buffer
+    for later processing.
+
+    Optionally a file path can be passed in and we will also
+    dump the characters to this file for debugging purposes.
+    """
     def __init__(self, address, file=None):
         self._recv_timeout_sec = 300
+        self._sleep_time = 0.5
         self._buffer = deque()
         self._asyncore_thread = None
         self._sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
@@ -70,31 +74,28 @@ class ConsoleSocket(asyncore.dispatcher):
 
     def handle_read(self):
         """process arriving characters into in memory _buffer"""
-        try:
-            data = asyncore.dispatcher.recv(self, 1)
-            # latin1 is needed since there are some chars
-            # we are receiving that cannot be encoded to utf-8
-            # such as 0xe2, 0x80, 0xA6.
-            string = data.decode("latin1")
-        except:
-            print("Exception seen.")
-            traceback.print_exc()
-            return
+        data = asyncore.dispatcher.recv(self, 1)
+        # latin1 is needed since there are some chars
+        # we are receiving that cannot be encoded to utf-8
+        # such as 0xe2, 0x80, 0xA6.
+        string = data.decode("latin1")
         if self._logfile:
             self._logfile.write("{}".format(string))
             self._logfile.flush()
         for c in string:
             self._buffer.extend(c)
 
-    def recv(self, n=1, sleep_delay_s=0.1):
-        """Return chars from in memory buffer"""
+    def recv(self, buffer_size=1):
+        """Return chars from in memory buffer.
+           Maintains the same API as socket.socket.recv.
+        """
         start_time = time.time()
-        while len(self._buffer) < n:
-            time.sleep(sleep_delay_s)
+        while len(self._buffer) < buffer_size:
+            time.sleep(self._sleep_time)
             elapsed_sec = time.time() - start_time
             if elapsed_sec > self._recv_timeout_sec:
                 raise socket.timeout
-        chars = ''.join([self._buffer.popleft() for i in range(n)])
+        chars = ''.join([self._buffer.popleft() for i in range(buffer_size)])
         # We choose to use latin1 to remain consistent with
         # handle_read() and give back the same data as the user would
         # receive if they were reading directly from the
diff --git a/python/qemu/machine.py b/python/qemu/machine.py
index 51aa255ef96..a09768e9f92 100644
--- a/python/qemu/machine.py
+++ b/python/qemu/machine.py
@@ -27,7 +27,7 @@ import socket
 import tempfile
 from typing import Optional, Type
 from types import TracebackType
-from qemu.console_socket import ConsoleSocket
+from . import console_socket
 
 from . import qmp
 
@@ -674,8 +674,9 @@ class QEMUMachine:
         """
         if self._console_socket is None:
             if self._drain_console:
-                self._console_socket = ConsoleSocket(self._console_address,
-                                                    file=self._console_log_path)
+                self._console_socket = console_socket.ConsoleSocket(
+                    self._console_address,
+                    file=self._console_log_path)
             else:
                 self._console_socket = socket.socket(socket.AF_UNIX,
                                                      socket.SOCK_STREAM)
diff --git a/python/qemu/pylintrc b/python/qemu/pylintrc
index 5d6ae7367d8..3f69205000d 100644
--- a/python/qemu/pylintrc
+++ b/python/qemu/pylintrc
@@ -33,7 +33,7 @@ good-names=i,
            Run,
            _,
            fd,
-
+           c,
 [VARIABLES]
 
 [STRING]
-- 
2.20.1

[PULL 15/16] python/qemu: Change ConsoleSocket to optionally drain socket.

[Alex Bennée]

From: Robert Foley <robert.foley@linaro.org>

The primary purpose of this change is to clean up
machine.py's console_socket property to return a single type,
a ConsoleSocket.

ConsoleSocket now derives from a socket, which means that
in the default case (of not draining), machine.py
will see the same behavior as it did prior to ConsoleSocket.

Signed-off-by: Robert Foley <robert.foley@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200717203041.9867-3-robert.foley@linaro.org>
Message-Id: <20200724064509.331-16-alex.bennee@linaro.org>

diff --git a/python/qemu/console_socket.py b/python/qemu/console_socket.py
index 09986bc2152..70869fbbdc4 100644
--- a/python/qemu/console_socket.py
+++ b/python/qemu/console_socket.py
@@ -13,68 +13,75 @@ which can drain a socket and optionally dump the bytes to file.
 # the COPYING file in the top-level directory.
 #
 
-import asyncore
 import socket
 import threading
 from collections import deque
 import time
 
 
-class ConsoleSocket(asyncore.dispatcher):
+class ConsoleSocket(socket.socket):
     """
     ConsoleSocket represents a socket attached to a char device.
 
-    Drains the socket and places the bytes into an in memory buffer
-    for later processing.
+    Optionally (if drain==True), drains the socket and places the bytes
+    into an in memory buffer for later processing.
 
     Optionally a file path can be passed in and we will also
     dump the characters to this file for debugging purposes.
     """
-    def __init__(self, address, file=None):
+    def __init__(self, address, file=None, drain=False):
         self._recv_timeout_sec = 300
         self._sleep_time = 0.5
         self._buffer = deque()
-        self._asyncore_thread = None
-        self._sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
-        self._sock.connect(address)
+        socket.socket.__init__(self, socket.AF_UNIX, socket.SOCK_STREAM)
+        self.connect(address)
         self._logfile = None
         if file:
             self._logfile = open(file, "w")
-        asyncore.dispatcher.__init__(self, sock=self._sock)
         self._open = True
-        self._thread_start()
+        if drain:
+            self._drain_thread = self._thread_start()
+        else:
+            self._drain_thread = None
 
-    def _thread_start(self):
-        """Kick off a thread to wait on the asyncore.loop"""
-        if self._asyncore_thread is not None:
-            return
-        self._asyncore_thread = threading.Thread(target=asyncore.loop,
-                                                 kwargs={'timeout':1})
-        self._asyncore_thread.daemon = True
-        self._asyncore_thread.start()
+    def _drain_fn(self):
+        """Drains the socket and runs while the socket is open."""
+        while self._open:
+            try:
+                self._drain_socket()
+            except socket.timeout:
+                # The socket is expected to timeout since we set a
+                # short timeout to allow the thread to exit when
+                # self._open is set to False.
+                time.sleep(self._sleep_time)
 
-    def handle_close(self):
-        """redirect close to base class"""
-        # Call the base class close, but not self.close() since
-        # handle_close() occurs in the context of the thread which
-        # self.close() attempts to join.
-        asyncore.dispatcher.close(self)
+    def _thread_start(self):
+        """Kick off a thread to drain the socket."""
+        # Configure socket to not block and timeout.
+        # This allows our drain thread to not block
+        # on recieve and exit smoothly.
+        socket.socket.setblocking(self, False)
+        socket.socket.settimeout(self, 1)
+        drain_thread = threading.Thread(target=self._drain_fn)
+        drain_thread.daemon = True
+        drain_thread.start()
+        return drain_thread
 
     def close(self):
         """Close the base object and wait for the thread to terminate"""
         if self._open:
             self._open = False
-            asyncore.dispatcher.close(self)
-            if self._asyncore_thread is not None:
-                thread, self._asyncore_thread = self._asyncore_thread, None
+            if self._drain_thread is not None:
+                thread, self._drain_thread = self._drain_thread, None
                 thread.join()
+            socket.socket.close(self)
             if self._logfile:
                 self._logfile.close()
                 self._logfile = None
 
-    def handle_read(self):
+    def _drain_socket(self):
         """process arriving characters into in memory _buffer"""
-        data = asyncore.dispatcher.recv(self, 1)
+        data = socket.socket.recv(self, 1)
         # latin1 is needed since there are some chars
         # we are receiving that cannot be encoded to utf-8
         # such as 0xe2, 0x80, 0xA6.
@@ -85,27 +92,38 @@ class ConsoleSocket(asyncore.dispatcher):
         for c in string:
             self._buffer.extend(c)
 
-    def recv(self, buffer_size=1):
+    def recv(self, bufsize=1):
         """Return chars from in memory buffer.
            Maintains the same API as socket.socket.recv.
         """
+        if self._drain_thread is None:
+            # Not buffering the socket, pass thru to socket.
+            return socket.socket.recv(self, bufsize)
         start_time = time.time()
-        while len(self._buffer) < buffer_size:
+        while len(self._buffer) < bufsize:
             time.sleep(self._sleep_time)
             elapsed_sec = time.time() - start_time
             if elapsed_sec > self._recv_timeout_sec:
                 raise socket.timeout
-        chars = ''.join([self._buffer.popleft() for i in range(buffer_size)])
+        chars = ''.join([self._buffer.popleft() for i in range(bufsize)])
         # We choose to use latin1 to remain consistent with
         # handle_read() and give back the same data as the user would
         # receive if they were reading directly from the
         # socket w/o our intervention.
         return chars.encode("latin1")
 
-    def set_blocking(self):
-        """Maintain compatibility with socket API"""
-        pass
+    def setblocking(self, value):
+        """When not draining we pass thru to the socket,
+           since when draining we control socket blocking.
+        """
+        if self._drain_thread is None:
+            socket.socket.setblocking(self, value)
 
     def settimeout(self, seconds):
-        """Set current timeout on recv"""
-        self._recv_timeout_sec = seconds
+        """When not draining we pass thru to the socket,
+           since when draining we control the timeout.
+        """
+        if seconds is not None:
+            self._recv_timeout_sec = seconds
+        if self._drain_thread is None:
+            socket.socket.settimeout(self, seconds)
diff --git a/python/qemu/machine.py b/python/qemu/machine.py
index a09768e9f92..82f3731fc3f 100644
--- a/python/qemu/machine.py
+++ b/python/qemu/machine.py
@@ -23,7 +23,6 @@ import os
 import subprocess
 import shutil
 import signal
-import socket
 import tempfile
 from typing import Optional, Type
 from types import TracebackType
@@ -673,12 +672,8 @@ class QEMUMachine:
         Returns a socket connected to the console
         """
         if self._console_socket is None:
-            if self._drain_console:
-                self._console_socket = console_socket.ConsoleSocket(
-                    self._console_address,
-                    file=self._console_log_path)
-            else:
-                self._console_socket = socket.socket(socket.AF_UNIX,
-                                                     socket.SOCK_STREAM)
-                self._console_socket.connect(self._console_address)
+            self._console_socket = console_socket.ConsoleSocket(
+                self._console_address,
+                file=self._console_log_path,
+                drain=self._drain_console)
         return self._console_socket
-- 
2.20.1

[PULL 16/16] tests/vm: add shutdown timeout in basevm.py

[Alex Bennée]

From: Robert Foley <robert.foley@linaro.org>

We are adding the shutdown timeout to solve an issue
we now see where the aarch64 VMs timeout on shutdown
under TCG.

There is a new 3 second timeout in machine.py,
which we override in basevm.py when shutting down.

Signed-off-by: Robert Foley <robert.foley@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200717203041.9867-4-robert.foley@linaro.org>
Message-Id: <20200724064509.331-17-alex.bennee@linaro.org>

diff --git a/tests/vm/basevm.py b/tests/vm/basevm.py
index 7acb48b8761..3fac20e929a 100644
--- a/tests/vm/basevm.py
+++ b/tests/vm/basevm.py
@@ -80,6 +80,8 @@ class BaseVM(object):
     arch = "#arch"
     # command to halt the guest, can be overridden by subclasses
     poweroff = "poweroff"
+    # Time to wait for shutdown to finish.
+    shutdown_timeout_default = 30
     # enable IPv6 networking
     ipv6 = True
     # This is the timeout on the wait for console bytes.
@@ -87,7 +89,7 @@ class BaseVM(object):
     # Scale up some timeouts under TCG.
     # 4 is arbitrary, but greater than 2,
     # since we found we need to wait more than twice as long.
-    tcg_ssh_timeout_multiplier = 4
+    tcg_timeout_multiplier = 4
     def __init__(self, args, config=None):
         self._guest = None
         self._genisoimage = args.genisoimage
@@ -141,9 +143,12 @@ class BaseVM(object):
         if args.jobs and args.jobs > 1:
             self._args += ["-smp", "%d" % args.jobs]
         if kvm_available(self.arch):
+            self._shutdown_timeout = self.shutdown_timeout_default
             self._args += ["-enable-kvm"]
         else:
             logging.info("KVM not available, not using -enable-kvm")
+            self._shutdown_timeout = \
+                self.shutdown_timeout_default * self.tcg_timeout_multiplier
         self._data_args = []
 
         if self._config['qemu_args'] != None:
@@ -423,7 +428,7 @@ class BaseVM(object):
     def wait_ssh(self, wait_root=False, seconds=300, cmd="exit 0"):
         # Allow more time for VM to boot under TCG.
         if not kvm_available(self.arch):
-            seconds *= self.tcg_ssh_timeout_multiplier
+            seconds *= self.tcg_timeout_multiplier
         starttime = datetime.datetime.now()
         endtime = starttime + datetime.timedelta(seconds=seconds)
         cmd_success = False
@@ -441,14 +446,14 @@ class BaseVM(object):
             raise Exception("Timeout while waiting for guest ssh")
 
     def shutdown(self):
-        self._guest.shutdown()
+        self._guest.shutdown(timeout=self._shutdown_timeout)
 
     def wait(self):
-        self._guest.wait()
+        self._guest.wait(timeout=self._shutdown_timeout)
 
     def graceful_shutdown(self):
         self.ssh_root(self.poweroff)
-        self._guest.wait()
+        self._guest.wait(timeout=self._shutdown_timeout)
 
     def qmp(self, *args, **kwargs):
         return self._guest.qmp(*args, **kwargs)
-- 
2.20.1

Re: [PULL 00/16] various fixes for rc2 (semihosting, tcg, docker, tests/vm)

[Peter Maydell]

On Mon, 27 Jul 2020 at 13:23, Alex Bennée <alex.bennee@linaro.org> wrote:
>
> The following changes since commit 194f8ca825854abef3aceca1ed7eb5a53b08751f:
>
>   Merge remote-tracking branch 'remotes/rth/tags/pull-tcg-20200725' into staging (2020-07-26 17:17:58 +0100)
>
> are available in the Git repository at:
>
>   https://github.com/stsquad/qemu.git tags/pull-fixes-for-rc2-270720-1
>
> for you to fetch changes up to 4a70232b1d26b0d73e1bce60b2c3bdb7e4279d16:
>
>   tests/vm: add shutdown timeout in basevm.py (2020-07-27 09:41:58 +0100)
>
> ----------------------------------------------------------------
> Various fixes for rc2:
>
>   - get shippable working again
>   - semihosting bug fixes
>   - tweak tb-size handling for low memory machines
>   - i386 compound literal float fix
>   - linux-user MAP_FIXED->MAP_NOREPLACE on fallback
>   - docker binfmt_misc fixes
>   - linux-user nanosleep fix
>   - tests/vm drain console fixes


Applied, thanks.

Please update the changelog at https://wiki.qemu.org/ChangeLog/5.1
for any user-visible changes.

-- PMM