[PATCH v3 00/16] candidate fixes for 5.1-rc1 (testing, semihosting, OOM tcg, x86 fpu)

[PATCH v3 00/16] candidate fixes for 5.1-rc1 (testing, semihosting, OOM tcg, x86 fpu)

[Alex Bennée]

Hi,

This is the current state of my candidate fixes for 5.1. Apart from
the revisions following reviews for v2:
  
  https://patchew.org/QEMU/20200722062902.24509-1-alex.bennee@linaro.org/

this includes a new version of Laurent's nanosleep fix and some
clean-ups to the python console drain code which should sort out some
of the 100% cpu spins Peter was seeing.

Other minor tweaks noted bellow -- in the patches themselves. The
following are still un-reviewed:

 - tests/docker: fix update command due to python3 str/bytes distinction
 - accel/tcg: better handle memory constrained systems
 - util/oslib-win32: add qemu_get_host_physmem implementation

Alex Bennée (8):
  shippable: add one more qemu to registry url
  util: add qemu_get_host_physmem utility function
  util/oslib-win32: add qemu_get_host_physmem implementation
  accel/tcg: better handle memory constrained systems
  linux-user: don't use MAP_FIXED in pgd_find_hole_fallback
  tests/docker: fix update command due to python3 str/bytes distinction
  tests/docker: fix binfmt_misc image building
  tests/docker: add support for DEB_KEYRING

KONRAD Frederic (2):
  semihosting: defer connect_chardevs a little more to use serialx
  semihosting: don't send the trailing '\0'

Laszlo Ersek (1):
  target/i386: floatx80: avoid compound literals in static initializers

Laurent Vivier (2):
  linux-user: fix clock_nanosleep()
  linux-user,ppc: fix clock_nanosleep() for linux-user-ppc

Robert Foley (3):
  python/qemu: Cleanup changes to ConsoleSocket
  python/qemu: Change ConsoleSocket to optionally drain socket.
  tests/vm: add shutdown timeout in basevm.py

 include/fpu/softfloat.h                       |   1 +
 include/qemu/osdep.h                          |  15 +
 accel/tcg/translate-all.c                     |   7 +-
 hw/semihosting/console.c                      |   4 +-
 linux-user/elfload.c                          |  10 +-
 linux-user/syscall.c                          |  15 +-
 softmmu/vl.c                                  |   5 +-
 target/i386/fpu_helper.c                      | 426 +++++++++---------
 util/oslib-posix.c                            |  15 +
 util/oslib-win32.c                            |  11 +
 .shippable.yml                                |   2 +-
 python/qemu/console_socket.py                 | 137 +++---
 python/qemu/machine.py                        |  14 +-
 python/qemu/pylintrc                          |   2 +-
 tests/docker/Makefile.include                 |   2 +-
 tests/docker/docker.py                        |  13 +-
 tests/docker/dockerfiles/debian-bootstrap.pre |   7 +
 tests/vm/basevm.py                            |  15 +-
 18 files changed, 390 insertions(+), 311 deletions(-)

-- 
2.20.1

[PATCH v3 01/16] shippable: add one more qemu to registry url

[Alex Bennée]

The registry url is <project>/<repo>/qemu/<image>

Perhaps we should rationalise that some day but for now.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20200717105139.25293-2-alex.bennee@linaro.org>
Message-Id: <20200722062902.24509-2-alex.bennee@linaro.org>
---
 .shippable.yml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/.shippable.yml b/.shippable.yml
index f6b742432e5..89d8be4291b 100644
--- a/.shippable.yml
+++ b/.shippable.yml
@@ -27,7 +27,7 @@ env:
       TARGET_LIST=ppc64-softmmu,ppc64-linux-user,ppc64abi32-linux-user
 build:
   pre_ci_boot:
-    image_name: registry.gitlab.com/qemu-project/qemu/${IMAGE}
+    image_name: registry.gitlab.com/qemu-project/qemu/qemu/${IMAGE}
     image_tag: latest
     pull: true
     options: "-e HOME=/root"
-- 
2.20.1

[PATCH v3 02/16] semihosting: defer connect_chardevs a little more to use serialx

[Alex Bennée]

From: KONRAD Frederic <frederic.konrad@adacore.com>

With that we can just use -semihosting-config chardev=serial0.

[AJB: tweak commit message]

Signed-off-by: KONRAD Frederic <frederic.konrad@adacore.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <1592215252-26742-1-git-send-email-frederic.konrad@adacore.com>
Message-Id: <20200717105139.25293-3-alex.bennee@linaro.org>
Message-Id: <20200722062902.24509-3-alex.bennee@linaro.org>
---
 softmmu/vl.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/softmmu/vl.c b/softmmu/vl.c
index 34162415579..295459adc95 100644
--- a/softmmu/vl.c
+++ b/softmmu/vl.c
@@ -4127,8 +4127,6 @@ void qemu_init(int argc, char **argv, char **envp)
 
     qemu_opts_foreach(qemu_find_opts("chardev"),
                       chardev_init_func, NULL, &error_fatal);
-    /* now chardevs have been created we may have semihosting to connect */
-    qemu_semihosting_connect_chardevs();
 
 #ifdef CONFIG_VIRTFS
     qemu_opts_foreach(qemu_find_opts("fsdev"),
@@ -4277,6 +4275,9 @@ void qemu_init(int argc, char **argv, char **envp)
     if (foreach_device_config(DEV_DEBUGCON, debugcon_parse) < 0)
         exit(1);
 
+    /* now chardevs have been created we may have semihosting to connect */
+    qemu_semihosting_connect_chardevs();
+
     /* If no default VGA is requested, the default is "none".  */
     if (default_vga) {
         vga_model = get_default_vga_model(machine_class);
-- 
2.20.1

[PATCH v3 03/16] semihosting: don't send the trailing '\0'

[Alex Bennée]

From: KONRAD Frederic <frederic.konrad@adacore.com>

Don't send the trailing 0 from the string.

Signed-off-by: KONRAD Frederic <frederic.konrad@adacore.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <1592215252-26742-2-git-send-email-frederic.konrad@adacore.com>
Message-Id: <20200717105139.25293-4-alex.bennee@linaro.org>
Message-Id: <20200722062902.24509-4-alex.bennee@linaro.org>
---
 hw/semihosting/console.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/hw/semihosting/console.c b/hw/semihosting/console.c
index 22e7827824a..9b4fee92602 100644
--- a/hw/semihosting/console.c
+++ b/hw/semihosting/console.c
@@ -52,7 +52,9 @@ static GString *copy_user_string(CPUArchState *env, target_ulong addr)
 
     do {
         if (cpu_memory_rw_debug(cpu, addr++, &c, 1, 0) == 0) {
-            s = g_string_append_c(s, c);
+            if (c) {
+                s = g_string_append_c(s, c);
+            }
         } else {
             qemu_log_mask(LOG_GUEST_ERROR,
                           "%s: passed inaccessible address " TARGET_FMT_lx,
-- 
2.20.1

[PATCH v3 04/16] util: add qemu_get_host_physmem utility function

[Alex Bennée]

This will be used in a future patch. For POSIX systems _SC_PHYS_PAGES
isn't standardised but at least appears in the man pages for
Open/FreeBSD. The result is advisory so any users of it shouldn't just
fail if we can't work it out.

The win32 stub currently returns 0 until someone with a Windows system
can develop and test a patch.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Cc: BALATON Zoltan <balaton@eik.bme.hu>
Cc: Christian Ehrhardt <christian.ehrhardt@canonical.com>
Message-Id: <20200722062902.24509-5-alex.bennee@linaro.org>
---
 include/qemu/osdep.h | 12 ++++++++++++
 util/oslib-posix.c   | 15 +++++++++++++++
 util/oslib-win32.c   |  6 ++++++
 3 files changed, 33 insertions(+)

diff --git a/include/qemu/osdep.h b/include/qemu/osdep.h
index 45c217aa280..0b1298b3c91 100644
--- a/include/qemu/osdep.h
+++ b/include/qemu/osdep.h
@@ -668,4 +668,16 @@ static inline void qemu_reset_optind(void)
  */
 char *qemu_get_host_name(Error **errp);
 
+/**
+ * qemu_get_host_physmem:
+ *
+ * Operating system agnostic way of querying host memory.
+ *
+ * Returns amount of physical memory on the system. This is purely
+ * advisery and may return 0 if we can't work it out. At the other
+ * end we saturate to SIZE_MAX if you are lucky enough to have that
+ * much memory.
+ */
+size_t qemu_get_host_physmem(void);
+
 #endif
diff --git a/util/oslib-posix.c b/util/oslib-posix.c
index d9236746248..ad8001a4ad8 100644
--- a/util/oslib-posix.c
+++ b/util/oslib-posix.c
@@ -841,3 +841,18 @@ char *qemu_get_host_name(Error **errp)
 
     return g_steal_pointer(&hostname);
 }
+
+size_t qemu_get_host_physmem(void)
+{
+#ifdef _SC_PHYS_PAGES
+    long pages = sysconf(_SC_PHYS_PAGES);
+    if (pages > 0) {
+        if (pages > SIZE_MAX / qemu_real_host_page_size) {
+            return SIZE_MAX;
+        } else {
+            return pages * qemu_real_host_page_size;
+        }
+    }
+#endif
+    return 0;
+}
diff --git a/util/oslib-win32.c b/util/oslib-win32.c
index 7eedbe5859a..31030463cc9 100644
--- a/util/oslib-win32.c
+++ b/util/oslib-win32.c
@@ -828,3 +828,9 @@ char *qemu_get_host_name(Error **errp)
 
     return g_utf16_to_utf8(tmp, size, NULL, NULL, NULL);
 }
+
+size_t qemu_get_host_physmem(void)
+{
+    /* currently unimplemented */
+    return 0;
+}
-- 
2.20.1

[PATCH v3 05/16] util/oslib-win32: add qemu_get_host_physmem implementation

[Alex Bennée]

Compile tested only.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Cc: Stefan Weil <sw@weilnetz.de>

---
v2
  - used more widely available GlobalMemoryStatusEx
  - dropped Phillipe's r-b due to the change
---
 util/oslib-win32.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/util/oslib-win32.c b/util/oslib-win32.c
index 31030463cc9..a74ee5a3aa7 100644
--- a/util/oslib-win32.c
+++ b/util/oslib-win32.c
@@ -831,6 +831,11 @@ char *qemu_get_host_name(Error **errp)
 
 size_t qemu_get_host_physmem(void)
 {
-    /* currently unimplemented */
+    MEMORYSTATUSEX statex;
+    statex.dwLength = sizeof(statex);
+
+    if (!GlobalMemoryStatusEx(&statex)) {
+        return statex.ullTotalPhys;
+    }
     return 0;
 }
-- 
2.20.1

Re: [PATCH v3 05/16] util/oslib-win32: add qemu_get_host_physmem implementation

[Stefan Weil]

Am 24.07.20 um 08:44 schrieb Alex Bennée:

> Compile tested only.
>
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> Cc: Stefan Weil <sw@weilnetz.de>
>
> ---
> v2
>   - used more widely available GlobalMemoryStatusEx
>   - dropped Phillipe's r-b due to the change
> ---
>  util/oslib-win32.c | 7 ++++++-
>  1 file changed, 6 insertions(+), 1 deletion(-)
>
> diff --git a/util/oslib-win32.c b/util/oslib-win32.c
> index 31030463cc9..a74ee5a3aa7 100644
> --- a/util/oslib-win32.c
> +++ b/util/oslib-win32.c
> @@ -831,6 +831,11 @@ char *qemu_get_host_name(Error **errp)
>  
>  size_t qemu_get_host_physmem(void)
>  {
> -    /* currently unimplemented */
> +    MEMORYSTATUSEX statex;
> +    statex.dwLength = sizeof(statex);
> +
> +    if (!GlobalMemoryStatusEx(&statex)) {
> +        return statex.ullTotalPhys;
> +    }
>      return 0;
>  }


Thanks.

The logic must be reversed, as GlobalMemoryStatusEx returns a nonzero
value when it succeeded.

When that is fixed, you may add

Reviewed-by: Stefan Weil <sw@weilnetz.de>

[PATCH v3 06/16] accel/tcg: better handle memory constrained systems

[Alex Bennée]

It turns out there are some 64 bit systems that have relatively low
amounts of physical memory available to them (typically CI system).
Even with swapping available a 1GB translation buffer that fills up
can put the machine under increased memory pressure. Detect these low
memory situations and reduce tb_size appropriately.

Fixes: 600e17b2615 ("accel/tcg: increase default code gen buffer size for 64 bit")
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Cc: BALATON Zoltan <balaton@eik.bme.hu>
Cc: Christian Ehrhardt <christian.ehrhardt@canonical.com>

---
v3
  - use slightly tweak the logic to taper off more nicely
---
 accel/tcg/translate-all.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c
index 2afa46bd2b1..2d83013633b 100644
--- a/accel/tcg/translate-all.c
+++ b/accel/tcg/translate-all.c
@@ -976,7 +976,12 @@ static inline size_t size_code_gen_buffer(size_t tb_size)
 {
     /* Size the buffer.  */
     if (tb_size == 0) {
-        tb_size = DEFAULT_CODE_GEN_BUFFER_SIZE;
+        size_t phys_mem = qemu_get_host_physmem();
+        if (phys_mem == 0) {
+            tb_size = DEFAULT_CODE_GEN_BUFFER_SIZE;
+        } else {
+            tb_size = MIN(DEFAULT_CODE_GEN_BUFFER_SIZE, phys_mem / 8);
+        }
     }
     if (tb_size < MIN_CODE_GEN_BUFFER_SIZE) {
         tb_size = MIN_CODE_GEN_BUFFER_SIZE;
-- 
2.20.1

Re: [PATCH v3 06/16] accel/tcg: better handle memory constrained systems

[Robert Foley]

Reviewed-by: Robert Foley <robert.foley@linaro.org>


On Fri, 24 Jul 2020 at 02:49, Alex Bennée <alex.bennee@linaro.org> wrote:
>
> It turns out there are some 64 bit systems that have relatively low
> amounts of physical memory available to them (typically CI system).
> Even with swapping available a 1GB translation buffer that fills up
> can put the machine under increased memory pressure. Detect these low
> memory situations and reduce tb_size appropriately.
>
> Fixes: 600e17b2615 ("accel/tcg: increase default code gen buffer size for 64 bit")
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> Cc: BALATON Zoltan <balaton@eik.bme.hu>
> Cc: Christian Ehrhardt <christian.ehrhardt@canonical.com>
>
> ---
> v3
>   - use slightly tweak the logic to taper off more nicely
> ---
>  accel/tcg/translate-all.c | 7 ++++++-
>  1 file changed, 6 insertions(+), 1 deletion(-)
>
> diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c
> index 2afa46bd2b1..2d83013633b 100644
> --- a/accel/tcg/translate-all.c
> +++ b/accel/tcg/translate-all.c
> @@ -976,7 +976,12 @@ static inline size_t size_code_gen_buffer(size_t tb_size)
>  {
>      /* Size the buffer.  */
>      if (tb_size == 0) {
> -        tb_size = DEFAULT_CODE_GEN_BUFFER_SIZE;
> +        size_t phys_mem = qemu_get_host_physmem();
> +        if (phys_mem == 0) {
> +            tb_size = DEFAULT_CODE_GEN_BUFFER_SIZE;
> +        } else {
> +            tb_size = MIN(DEFAULT_CODE_GEN_BUFFER_SIZE, phys_mem / 8);
> +        }
>      }
>      if (tb_size < MIN_CODE_GEN_BUFFER_SIZE) {
>          tb_size = MIN_CODE_GEN_BUFFER_SIZE;
> --
> 2.20.1
>
>

Re: [PATCH v3 06/16] accel/tcg: better handle memory constrained systems

[Richard Henderson]

On 7/23/20 11:44 PM, Alex Bennée wrote:
> It turns out there are some 64 bit systems that have relatively low
> amounts of physical memory available to them (typically CI system).
> Even with swapping available a 1GB translation buffer that fills up
> can put the machine under increased memory pressure. Detect these low
> memory situations and reduce tb_size appropriately.
> 
> Fixes: 600e17b2615 ("accel/tcg: increase default code gen buffer size for 64 bit")
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> Cc: BALATON Zoltan <balaton@eik.bme.hu>
> Cc: Christian Ehrhardt <christian.ehrhardt@canonical.com>

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>

r~

[PATCH v3 07/16] target/i386: floatx80: avoid compound literals in static initializers

[Alex Bennée]

From: Laszlo Ersek <lersek@redhat.com>

Quoting ISO C99 6.7.8p4, "All the expressions in an initializer for an
object that has static storage duration shall be constant expressions or
string literals".

The compound literal produced by the make_floatx80() macro is not such a
constant expression, per 6.6p7-9. (An implementation may accept it,
according to 6.6p10, but is not required to.)

Therefore using "floatx80_zero" and make_floatx80() for initializing
"f2xm1_table" and "fpatan_table" is not portable. And gcc-4.8 in RHEL-7.6
actually chokes on them:

> target/i386/fpu_helper.c:871:5: error: initializer element is not constant
>      { make_floatx80(0xbfff, 0x8000000000000000ULL),
>      ^

We've had the make_floatx80_init() macro for this purpose since commit
3bf7e40ab914 ("softfloat: fix for C99", 2012-03-17), so let's use that
macro again.

Fixes: eca30647fc07
Fixes: ff57bb7b6326
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Cc: Aurelien Jarno <aurelien@aurel32.net>
Cc: Eduardo Habkost <ehabkost@redhat.com>
Cc: Joseph Myers <joseph@codesourcery.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Peter Maydell <peter.maydell@linaro.org>
Cc: Richard Henderson <rth@twiddle.net>
Link: https://lists.gnu.org/archive/html/qemu-devel/2017-08/msg06566.html
Link: https://lists.gnu.org/archive/html/qemu-devel/2020-07/msg04714.html
Message-Id: <20200716144251.23004-1-lersek@redhat.com>
Message-Id: <20200722062902.24509-8-alex.bennee@linaro.org>
---
 include/fpu/softfloat.h  |   1 +
 target/i386/fpu_helper.c | 426 +++++++++++++++++++--------------------
 2 files changed, 214 insertions(+), 213 deletions(-)

diff --git a/include/fpu/softfloat.h b/include/fpu/softfloat.h
index f1a19df066b..659218b5c78 100644
--- a/include/fpu/softfloat.h
+++ b/include/fpu/softfloat.h
@@ -822,6 +822,7 @@ static inline bool floatx80_invalid_encoding(floatx80 a)
 }
 
 #define floatx80_zero make_floatx80(0x0000, 0x0000000000000000LL)
+#define floatx80_zero_init make_floatx80_init(0x0000, 0x0000000000000000LL)
 #define floatx80_one make_floatx80(0x3fff, 0x8000000000000000LL)
 #define floatx80_ln2 make_floatx80(0x3ffe, 0xb17217f7d1cf79acLL)
 #define floatx80_pi make_floatx80(0x4000, 0xc90fdaa22168c235LL)
diff --git a/target/i386/fpu_helper.c b/target/i386/fpu_helper.c
index f5e6c4b88d4..4ea73874d83 100644
--- a/target/i386/fpu_helper.c
+++ b/target/i386/fpu_helper.c
@@ -868,201 +868,201 @@ struct f2xm1_data {
 };
 
 static const struct f2xm1_data f2xm1_table[65] = {
-    { make_floatx80(0xbfff, 0x8000000000000000ULL),
-      make_floatx80(0x3ffe, 0x8000000000000000ULL),
-      make_floatx80(0xbffe, 0x8000000000000000ULL) },
-    { make_floatx80(0xbffe, 0xf800000000002e7eULL),
-      make_floatx80(0x3ffe, 0x82cd8698ac2b9160ULL),
-      make_floatx80(0xbffd, 0xfa64f2cea7a8dd40ULL) },
-    { make_floatx80(0xbffe, 0xefffffffffffe960ULL),
-      make_floatx80(0x3ffe, 0x85aac367cc488345ULL),
-      make_floatx80(0xbffd, 0xf4aa7930676ef976ULL) },
-    { make_floatx80(0xbffe, 0xe800000000006f10ULL),
-      make_floatx80(0x3ffe, 0x88980e8092da5c14ULL),
-      make_floatx80(0xbffd, 0xeecfe2feda4b47d8ULL) },
-    { make_floatx80(0xbffe, 0xe000000000008a45ULL),
-      make_floatx80(0x3ffe, 0x8b95c1e3ea8ba2a5ULL),
-      make_floatx80(0xbffd, 0xe8d47c382ae8bab6ULL) },
-    { make_floatx80(0xbffe, 0xd7ffffffffff8a9eULL),
-      make_floatx80(0x3ffe, 0x8ea4398b45cd8116ULL),
-      make_floatx80(0xbffd, 0xe2b78ce97464fdd4ULL) },
-    { make_floatx80(0xbffe, 0xd0000000000019a0ULL),
-      make_floatx80(0x3ffe, 0x91c3d373ab11b919ULL),
-      make_floatx80(0xbffd, 0xdc785918a9dc8dceULL) },
-    { make_floatx80(0xbffe, 0xc7ffffffffff14dfULL),
-      make_floatx80(0x3ffe, 0x94f4efa8fef76836ULL),
-      make_floatx80(0xbffd, 0xd61620ae02112f94ULL) },
-    { make_floatx80(0xbffe, 0xc000000000006530ULL),
-      make_floatx80(0x3ffe, 0x9837f0518db87fbbULL),
-      make_floatx80(0xbffd, 0xcf901f5ce48f008aULL) },
-    { make_floatx80(0xbffe, 0xb7ffffffffff1723ULL),
-      make_floatx80(0x3ffe, 0x9b8d39b9d54eb74cULL),
-      make_floatx80(0xbffd, 0xc8e58c8c55629168ULL) },
-    { make_floatx80(0xbffe, 0xb00000000000b5e1ULL),
-      make_floatx80(0x3ffe, 0x9ef5326091a0c366ULL),
-      make_floatx80(0xbffd, 0xc2159b3edcbe7934ULL) },
-    { make_floatx80(0xbffe, 0xa800000000006f8aULL),
-      make_floatx80(0x3ffe, 0xa27043030c49370aULL),
-      make_floatx80(0xbffd, 0xbb1f79f9e76d91ecULL) },
-    { make_floatx80(0xbffe, 0x9fffffffffff816aULL),
-      make_floatx80(0x3ffe, 0xa5fed6a9b15171cfULL),
-      make_floatx80(0xbffd, 0xb40252ac9d5d1c62ULL) },
-    { make_floatx80(0xbffe, 0x97ffffffffffb621ULL),
-      make_floatx80(0x3ffe, 0xa9a15ab4ea7c30e6ULL),
-      make_floatx80(0xbffd, 0xacbd4a962b079e34ULL) },
-    { make_floatx80(0xbffe, 0x8fffffffffff162bULL),
-      make_floatx80(0x3ffe, 0xad583eea42a1b886ULL),
-      make_floatx80(0xbffd, 0xa54f822b7abc8ef4ULL) },
-    { make_floatx80(0xbffe, 0x87ffffffffff4d34ULL),
-      make_floatx80(0x3ffe, 0xb123f581d2ac7b51ULL),
-      make_floatx80(0xbffd, 0x9db814fc5aa7095eULL) },
-    { make_floatx80(0xbffe, 0x800000000000227dULL),
-      make_floatx80(0x3ffe, 0xb504f333f9de539dULL),
-      make_floatx80(0xbffd, 0x95f619980c4358c6ULL) },
-    { make_floatx80(0xbffd, 0xefffffffffff3978ULL),
-      make_floatx80(0x3ffe, 0xb8fbaf4762fbd0a1ULL),
-      make_floatx80(0xbffd, 0x8e08a1713a085ebeULL) },
-    { make_floatx80(0xbffd, 0xe00000000000df81ULL),
-      make_floatx80(0x3ffe, 0xbd08a39f580bfd8cULL),
-      make_floatx80(0xbffd, 0x85eeb8c14fe804e8ULL) },
-    { make_floatx80(0xbffd, 0xd00000000000bccfULL),
-      make_floatx80(0x3ffe, 0xc12c4cca667062f6ULL),
-      make_floatx80(0xbffc, 0xfb4eccd6663e7428ULL) },
-    { make_floatx80(0xbffd, 0xc00000000000eff0ULL),
-      make_floatx80(0x3ffe, 0xc5672a1155069abeULL),
-      make_floatx80(0xbffc, 0xea6357baabe59508ULL) },
-    { make_floatx80(0xbffd, 0xb000000000000fe6ULL),
-      make_floatx80(0x3ffe, 0xc9b9bd866e2f234bULL),
-      make_floatx80(0xbffc, 0xd91909e6474372d4ULL) },
-    { make_floatx80(0xbffd, 0x9fffffffffff2172ULL),
-      make_floatx80(0x3ffe, 0xce248c151f84bf00ULL),
-      make_floatx80(0xbffc, 0xc76dcfab81ed0400ULL) },
-    { make_floatx80(0xbffd, 0x8fffffffffffafffULL),
-      make_floatx80(0x3ffe, 0xd2a81d91f12afb2bULL),
-      make_floatx80(0xbffc, 0xb55f89b83b541354ULL) },
-    { make_floatx80(0xbffc, 0xffffffffffff81a3ULL),
-      make_floatx80(0x3ffe, 0xd744fccad69d7d5eULL),
-      make_floatx80(0xbffc, 0xa2ec0cd4a58a0a88ULL) },
-    { make_floatx80(0xbffc, 0xdfffffffffff1568ULL),
-      make_floatx80(0x3ffe, 0xdbfbb797daf25a44ULL),
-      make_floatx80(0xbffc, 0x901121a0943696f0ULL) },
-    { make_floatx80(0xbffc, 0xbfffffffffff68daULL),
-      make_floatx80(0x3ffe, 0xe0ccdeec2a94f811ULL),
-      make_floatx80(0xbffb, 0xf999089eab583f78ULL) },
-    { make_floatx80(0xbffc, 0x9fffffffffff4690ULL),
-      make_floatx80(0x3ffe, 0xe5b906e77c83657eULL),
-      make_floatx80(0xbffb, 0xd237c8c41be4d410ULL) },
-    { make_floatx80(0xbffb, 0xffffffffffff8aeeULL),
-      make_floatx80(0x3ffe, 0xeac0c6e7dd24427cULL),
-      make_floatx80(0xbffb, 0xa9f9c8c116ddec20ULL) },
-    { make_floatx80(0xbffb, 0xbfffffffffff2d18ULL),
-      make_floatx80(0x3ffe, 0xefe4b99bdcdb06ebULL),
-      make_floatx80(0xbffb, 0x80da33211927c8a8ULL) },
-    { make_floatx80(0xbffa, 0xffffffffffff8ccbULL),
-      make_floatx80(0x3ffe, 0xf5257d152486d0f4ULL),
-      make_floatx80(0xbffa, 0xada82eadb792f0c0ULL) },
-    { make_floatx80(0xbff9, 0xffffffffffff11feULL),
-      make_floatx80(0x3ffe, 0xfa83b2db722a0846ULL),
-      make_floatx80(0xbff9, 0xaf89a491babef740ULL) },
-    { floatx80_zero,
-      make_floatx80(0x3fff, 0x8000000000000000ULL),
-      floatx80_zero },
-    { make_floatx80(0x3ff9, 0xffffffffffff2680ULL),
-      make_floatx80(0x3fff, 0x82cd8698ac2b9f6fULL),
-      make_floatx80(0x3ff9, 0xb361a62b0ae7dbc0ULL) },
-    { make_floatx80(0x3ffb, 0x800000000000b500ULL),
-      make_floatx80(0x3fff, 0x85aac367cc488345ULL),
-      make_floatx80(0x3ffa, 0xb5586cf9891068a0ULL) },
-    { make_floatx80(0x3ffb, 0xbfffffffffff4b67ULL),
-      make_floatx80(0x3fff, 0x88980e8092da7cceULL),
-      make_floatx80(0x3ffb, 0x8980e8092da7cce0ULL) },
-    { make_floatx80(0x3ffb, 0xffffffffffffff57ULL),
-      make_floatx80(0x3fff, 0x8b95c1e3ea8bd6dfULL),
-      make_floatx80(0x3ffb, 0xb95c1e3ea8bd6df0ULL) },
-    { make_floatx80(0x3ffc, 0x9fffffffffff811fULL),
-      make_floatx80(0x3fff, 0x8ea4398b45cd4780ULL),
-      make_floatx80(0x3ffb, 0xea4398b45cd47800ULL) },
-    { make_floatx80(0x3ffc, 0xbfffffffffff9980ULL),
-      make_floatx80(0x3fff, 0x91c3d373ab11b919ULL),
-      make_floatx80(0x3ffc, 0x8e1e9b9d588dc8c8ULL) },
-    { make_floatx80(0x3ffc, 0xdffffffffffff631ULL),
-      make_floatx80(0x3fff, 0x94f4efa8fef70864ULL),
-      make_floatx80(0x3ffc, 0xa7a77d47f7b84320ULL) },
-    { make_floatx80(0x3ffc, 0xffffffffffff2499ULL),
-      make_floatx80(0x3fff, 0x9837f0518db892d4ULL),
-      make_floatx80(0x3ffc, 0xc1bf828c6dc496a0ULL) },
-    { make_floatx80(0x3ffd, 0x8fffffffffff80fbULL),
-      make_floatx80(0x3fff, 0x9b8d39b9d54e3a79ULL),
-      make_floatx80(0x3ffc, 0xdc69cdceaa71d3c8ULL) },
-    { make_floatx80(0x3ffd, 0x9fffffffffffbc23ULL),
-      make_floatx80(0x3fff, 0x9ef5326091a10313ULL),
-      make_floatx80(0x3ffc, 0xf7a993048d081898ULL) },
-    { make_floatx80(0x3ffd, 0xafffffffffff20ecULL),
-      make_floatx80(0x3fff, 0xa27043030c49370aULL),
-      make_floatx80(0x3ffd, 0x89c10c0c3124dc28ULL) },
-    { make_floatx80(0x3ffd, 0xc00000000000fd2cULL),
-      make_floatx80(0x3fff, 0xa5fed6a9b15171cfULL),
-      make_floatx80(0x3ffd, 0x97fb5aa6c545c73cULL) },
-    { make_floatx80(0x3ffd, 0xd0000000000093beULL),
-      make_floatx80(0x3fff, 0xa9a15ab4ea7c30e6ULL),
-      make_floatx80(0x3ffd, 0xa6856ad3a9f0c398ULL) },
-    { make_floatx80(0x3ffd, 0xe00000000000c2aeULL),
-      make_floatx80(0x3fff, 0xad583eea42a17876ULL),
-      make_floatx80(0x3ffd, 0xb560fba90a85e1d8ULL) },
-    { make_floatx80(0x3ffd, 0xefffffffffff1e3fULL),
-      make_floatx80(0x3fff, 0xb123f581d2abef6cULL),
-      make_floatx80(0x3ffd, 0xc48fd6074aafbdb0ULL) },
-    { make_floatx80(0x3ffd, 0xffffffffffff1c23ULL),
-      make_floatx80(0x3fff, 0xb504f333f9de2cadULL),
-      make_floatx80(0x3ffd, 0xd413cccfe778b2b4ULL) },
-    { make_floatx80(0x3ffe, 0x8800000000006344ULL),
-      make_floatx80(0x3fff, 0xb8fbaf4762fbd0a1ULL),
-      make_floatx80(0x3ffd, 0xe3eebd1d8bef4284ULL) },
-    { make_floatx80(0x3ffe, 0x9000000000005d67ULL),
-      make_floatx80(0x3fff, 0xbd08a39f580c668dULL),
-      make_floatx80(0x3ffd, 0xf4228e7d60319a34ULL) },
-    { make_floatx80(0x3ffe, 0x9800000000009127ULL),
-      make_floatx80(0x3fff, 0xc12c4cca6670e042ULL),
-      make_floatx80(0x3ffe, 0x82589994cce1c084ULL) },
-    { make_floatx80(0x3ffe, 0x9fffffffffff06f9ULL),
-      make_floatx80(0x3fff, 0xc5672a11550655c3ULL),
-      make_floatx80(0x3ffe, 0x8ace5422aa0cab86ULL) },
-    { make_floatx80(0x3ffe, 0xa7fffffffffff80dULL),
-      make_floatx80(0x3fff, 0xc9b9bd866e2f234bULL),
-      make_floatx80(0x3ffe, 0x93737b0cdc5e4696ULL) },
-    { make_floatx80(0x3ffe, 0xafffffffffff1470ULL),
-      make_floatx80(0x3fff, 0xce248c151f83fd69ULL),
-      make_floatx80(0x3ffe, 0x9c49182a3f07fad2ULL) },
-    { make_floatx80(0x3ffe, 0xb800000000000e0aULL),
-      make_floatx80(0x3fff, 0xd2a81d91f12aec5cULL),
-      make_floatx80(0x3ffe, 0xa5503b23e255d8b8ULL) },
-    { make_floatx80(0x3ffe, 0xc00000000000b7faULL),
-      make_floatx80(0x3fff, 0xd744fccad69dd630ULL),
-      make_floatx80(0x3ffe, 0xae89f995ad3bac60ULL) },
-    { make_floatx80(0x3ffe, 0xc800000000003aa6ULL),
-      make_floatx80(0x3fff, 0xdbfbb797daf25a44ULL),
-      make_floatx80(0x3ffe, 0xb7f76f2fb5e4b488ULL) },
-    { make_floatx80(0x3ffe, 0xd00000000000a6aeULL),
-      make_floatx80(0x3fff, 0xe0ccdeec2a954685ULL),
-      make_floatx80(0x3ffe, 0xc199bdd8552a8d0aULL) },
-    { make_floatx80(0x3ffe, 0xd800000000004165ULL),
-      make_floatx80(0x3fff, 0xe5b906e77c837155ULL),
-      make_floatx80(0x3ffe, 0xcb720dcef906e2aaULL) },
-    { make_floatx80(0x3ffe, 0xe00000000000582cULL),
-      make_floatx80(0x3fff, 0xeac0c6e7dd24713aULL),
-      make_floatx80(0x3ffe, 0xd5818dcfba48e274ULL) },
-    { make_floatx80(0x3ffe, 0xe800000000001a5dULL),
-      make_floatx80(0x3fff, 0xefe4b99bdcdb06ebULL),
-      make_floatx80(0x3ffe, 0xdfc97337b9b60dd6ULL) },
-    { make_floatx80(0x3ffe, 0xefffffffffffc1efULL),
-      make_floatx80(0x3fff, 0xf5257d152486a2faULL),
-      make_floatx80(0x3ffe, 0xea4afa2a490d45f4ULL) },
-    { make_floatx80(0x3ffe, 0xf800000000001069ULL),
-      make_floatx80(0x3fff, 0xfa83b2db722a0e5cULL),
-      make_floatx80(0x3ffe, 0xf50765b6e4541cb8ULL) },
-    { make_floatx80(0x3fff, 0x8000000000000000ULL),
-      make_floatx80(0x4000, 0x8000000000000000ULL),
-      make_floatx80(0x3fff, 0x8000000000000000ULL) },
+    { make_floatx80_init(0xbfff, 0x8000000000000000ULL),
+      make_floatx80_init(0x3ffe, 0x8000000000000000ULL),
+      make_floatx80_init(0xbffe, 0x8000000000000000ULL) },
+    { make_floatx80_init(0xbffe, 0xf800000000002e7eULL),
+      make_floatx80_init(0x3ffe, 0x82cd8698ac2b9160ULL),
+      make_floatx80_init(0xbffd, 0xfa64f2cea7a8dd40ULL) },
+    { make_floatx80_init(0xbffe, 0xefffffffffffe960ULL),
+      make_floatx80_init(0x3ffe, 0x85aac367cc488345ULL),
+      make_floatx80_init(0xbffd, 0xf4aa7930676ef976ULL) },
+    { make_floatx80_init(0xbffe, 0xe800000000006f10ULL),
+      make_floatx80_init(0x3ffe, 0x88980e8092da5c14ULL),
+      make_floatx80_init(0xbffd, 0xeecfe2feda4b47d8ULL) },
+    { make_floatx80_init(0xbffe, 0xe000000000008a45ULL),
+      make_floatx80_init(0x3ffe, 0x8b95c1e3ea8ba2a5ULL),
+      make_floatx80_init(0xbffd, 0xe8d47c382ae8bab6ULL) },
+    { make_floatx80_init(0xbffe, 0xd7ffffffffff8a9eULL),
+      make_floatx80_init(0x3ffe, 0x8ea4398b45cd8116ULL),
+      make_floatx80_init(0xbffd, 0xe2b78ce97464fdd4ULL) },
+    { make_floatx80_init(0xbffe, 0xd0000000000019a0ULL),
+      make_floatx80_init(0x3ffe, 0x91c3d373ab11b919ULL),
+      make_floatx80_init(0xbffd, 0xdc785918a9dc8dceULL) },
+    { make_floatx80_init(0xbffe, 0xc7ffffffffff14dfULL),
+      make_floatx80_init(0x3ffe, 0x94f4efa8fef76836ULL),
+      make_floatx80_init(0xbffd, 0xd61620ae02112f94ULL) },
+    { make_floatx80_init(0xbffe, 0xc000000000006530ULL),
+      make_floatx80_init(0x3ffe, 0x9837f0518db87fbbULL),
+      make_floatx80_init(0xbffd, 0xcf901f5ce48f008aULL) },
+    { make_floatx80_init(0xbffe, 0xb7ffffffffff1723ULL),
+      make_floatx80_init(0x3ffe, 0x9b8d39b9d54eb74cULL),
+      make_floatx80_init(0xbffd, 0xc8e58c8c55629168ULL) },
+    { make_floatx80_init(0xbffe, 0xb00000000000b5e1ULL),
+      make_floatx80_init(0x3ffe, 0x9ef5326091a0c366ULL),
+      make_floatx80_init(0xbffd, 0xc2159b3edcbe7934ULL) },
+    { make_floatx80_init(0xbffe, 0xa800000000006f8aULL),
+      make_floatx80_init(0x3ffe, 0xa27043030c49370aULL),
+      make_floatx80_init(0xbffd, 0xbb1f79f9e76d91ecULL) },
+    { make_floatx80_init(0xbffe, 0x9fffffffffff816aULL),
+      make_floatx80_init(0x3ffe, 0xa5fed6a9b15171cfULL),
+      make_floatx80_init(0xbffd, 0xb40252ac9d5d1c62ULL) },
+    { make_floatx80_init(0xbffe, 0x97ffffffffffb621ULL),
+      make_floatx80_init(0x3ffe, 0xa9a15ab4ea7c30e6ULL),
+      make_floatx80_init(0xbffd, 0xacbd4a962b079e34ULL) },
+    { make_floatx80_init(0xbffe, 0x8fffffffffff162bULL),
+      make_floatx80_init(0x3ffe, 0xad583eea42a1b886ULL),
+      make_floatx80_init(0xbffd, 0xa54f822b7abc8ef4ULL) },
+    { make_floatx80_init(0xbffe, 0x87ffffffffff4d34ULL),
+      make_floatx80_init(0x3ffe, 0xb123f581d2ac7b51ULL),
+      make_floatx80_init(0xbffd, 0x9db814fc5aa7095eULL) },
+    { make_floatx80_init(0xbffe, 0x800000000000227dULL),
+      make_floatx80_init(0x3ffe, 0xb504f333f9de539dULL),
+      make_floatx80_init(0xbffd, 0x95f619980c4358c6ULL) },
+    { make_floatx80_init(0xbffd, 0xefffffffffff3978ULL),
+      make_floatx80_init(0x3ffe, 0xb8fbaf4762fbd0a1ULL),
+      make_floatx80_init(0xbffd, 0x8e08a1713a085ebeULL) },
+    { make_floatx80_init(0xbffd, 0xe00000000000df81ULL),
+      make_floatx80_init(0x3ffe, 0xbd08a39f580bfd8cULL),
+      make_floatx80_init(0xbffd, 0x85eeb8c14fe804e8ULL) },
+    { make_floatx80_init(0xbffd, 0xd00000000000bccfULL),
+      make_floatx80_init(0x3ffe, 0xc12c4cca667062f6ULL),
+      make_floatx80_init(0xbffc, 0xfb4eccd6663e7428ULL) },
+    { make_floatx80_init(0xbffd, 0xc00000000000eff0ULL),
+      make_floatx80_init(0x3ffe, 0xc5672a1155069abeULL),
+      make_floatx80_init(0xbffc, 0xea6357baabe59508ULL) },
+    { make_floatx80_init(0xbffd, 0xb000000000000fe6ULL),
+      make_floatx80_init(0x3ffe, 0xc9b9bd866e2f234bULL),
+      make_floatx80_init(0xbffc, 0xd91909e6474372d4ULL) },
+    { make_floatx80_init(0xbffd, 0x9fffffffffff2172ULL),
+      make_floatx80_init(0x3ffe, 0xce248c151f84bf00ULL),
+      make_floatx80_init(0xbffc, 0xc76dcfab81ed0400ULL) },
+    { make_floatx80_init(0xbffd, 0x8fffffffffffafffULL),
+      make_floatx80_init(0x3ffe, 0xd2a81d91f12afb2bULL),
+      make_floatx80_init(0xbffc, 0xb55f89b83b541354ULL) },
+    { make_floatx80_init(0xbffc, 0xffffffffffff81a3ULL),
+      make_floatx80_init(0x3ffe, 0xd744fccad69d7d5eULL),
+      make_floatx80_init(0xbffc, 0xa2ec0cd4a58a0a88ULL) },
+    { make_floatx80_init(0xbffc, 0xdfffffffffff1568ULL),
+      make_floatx80_init(0x3ffe, 0xdbfbb797daf25a44ULL),
+      make_floatx80_init(0xbffc, 0x901121a0943696f0ULL) },
+    { make_floatx80_init(0xbffc, 0xbfffffffffff68daULL),
+      make_floatx80_init(0x3ffe, 0xe0ccdeec2a94f811ULL),
+      make_floatx80_init(0xbffb, 0xf999089eab583f78ULL) },
+    { make_floatx80_init(0xbffc, 0x9fffffffffff4690ULL),
+      make_floatx80_init(0x3ffe, 0xe5b906e77c83657eULL),
+      make_floatx80_init(0xbffb, 0xd237c8c41be4d410ULL) },
+    { make_floatx80_init(0xbffb, 0xffffffffffff8aeeULL),
+      make_floatx80_init(0x3ffe, 0xeac0c6e7dd24427cULL),
+      make_floatx80_init(0xbffb, 0xa9f9c8c116ddec20ULL) },
+    { make_floatx80_init(0xbffb, 0xbfffffffffff2d18ULL),
+      make_floatx80_init(0x3ffe, 0xefe4b99bdcdb06ebULL),
+      make_floatx80_init(0xbffb, 0x80da33211927c8a8ULL) },
+    { make_floatx80_init(0xbffa, 0xffffffffffff8ccbULL),
+      make_floatx80_init(0x3ffe, 0xf5257d152486d0f4ULL),
+      make_floatx80_init(0xbffa, 0xada82eadb792f0c0ULL) },
+    { make_floatx80_init(0xbff9, 0xffffffffffff11feULL),
+      make_floatx80_init(0x3ffe, 0xfa83b2db722a0846ULL),
+      make_floatx80_init(0xbff9, 0xaf89a491babef740ULL) },
+    { floatx80_zero_init,
+      make_floatx80_init(0x3fff, 0x8000000000000000ULL),
+      floatx80_zero_init },
+    { make_floatx80_init(0x3ff9, 0xffffffffffff2680ULL),
+      make_floatx80_init(0x3fff, 0x82cd8698ac2b9f6fULL),
+      make_floatx80_init(0x3ff9, 0xb361a62b0ae7dbc0ULL) },
+    { make_floatx80_init(0x3ffb, 0x800000000000b500ULL),
+      make_floatx80_init(0x3fff, 0x85aac367cc488345ULL),
+      make_floatx80_init(0x3ffa, 0xb5586cf9891068a0ULL) },
+    { make_floatx80_init(0x3ffb, 0xbfffffffffff4b67ULL),
+      make_floatx80_init(0x3fff, 0x88980e8092da7cceULL),
+      make_floatx80_init(0x3ffb, 0x8980e8092da7cce0ULL) },
+    { make_floatx80_init(0x3ffb, 0xffffffffffffff57ULL),
+      make_floatx80_init(0x3fff, 0x8b95c1e3ea8bd6dfULL),
+      make_floatx80_init(0x3ffb, 0xb95c1e3ea8bd6df0ULL) },
+    { make_floatx80_init(0x3ffc, 0x9fffffffffff811fULL),
+      make_floatx80_init(0x3fff, 0x8ea4398b45cd4780ULL),
+      make_floatx80_init(0x3ffb, 0xea4398b45cd47800ULL) },
+    { make_floatx80_init(0x3ffc, 0xbfffffffffff9980ULL),
+      make_floatx80_init(0x3fff, 0x91c3d373ab11b919ULL),
+      make_floatx80_init(0x3ffc, 0x8e1e9b9d588dc8c8ULL) },
+    { make_floatx80_init(0x3ffc, 0xdffffffffffff631ULL),
+      make_floatx80_init(0x3fff, 0x94f4efa8fef70864ULL),
+      make_floatx80_init(0x3ffc, 0xa7a77d47f7b84320ULL) },
+    { make_floatx80_init(0x3ffc, 0xffffffffffff2499ULL),
+      make_floatx80_init(0x3fff, 0x9837f0518db892d4ULL),
+      make_floatx80_init(0x3ffc, 0xc1bf828c6dc496a0ULL) },
+    { make_floatx80_init(0x3ffd, 0x8fffffffffff80fbULL),
+      make_floatx80_init(0x3fff, 0x9b8d39b9d54e3a79ULL),
+      make_floatx80_init(0x3ffc, 0xdc69cdceaa71d3c8ULL) },
+    { make_floatx80_init(0x3ffd, 0x9fffffffffffbc23ULL),
+      make_floatx80_init(0x3fff, 0x9ef5326091a10313ULL),
+      make_floatx80_init(0x3ffc, 0xf7a993048d081898ULL) },
+    { make_floatx80_init(0x3ffd, 0xafffffffffff20ecULL),
+      make_floatx80_init(0x3fff, 0xa27043030c49370aULL),
+      make_floatx80_init(0x3ffd, 0x89c10c0c3124dc28ULL) },
+    { make_floatx80_init(0x3ffd, 0xc00000000000fd2cULL),
+      make_floatx80_init(0x3fff, 0xa5fed6a9b15171cfULL),
+      make_floatx80_init(0x3ffd, 0x97fb5aa6c545c73cULL) },
+    { make_floatx80_init(0x3ffd, 0xd0000000000093beULL),
+      make_floatx80_init(0x3fff, 0xa9a15ab4ea7c30e6ULL),
+      make_floatx80_init(0x3ffd, 0xa6856ad3a9f0c398ULL) },
+    { make_floatx80_init(0x3ffd, 0xe00000000000c2aeULL),
+      make_floatx80_init(0x3fff, 0xad583eea42a17876ULL),
+      make_floatx80_init(0x3ffd, 0xb560fba90a85e1d8ULL) },
+    { make_floatx80_init(0x3ffd, 0xefffffffffff1e3fULL),
+      make_floatx80_init(0x3fff, 0xb123f581d2abef6cULL),
+      make_floatx80_init(0x3ffd, 0xc48fd6074aafbdb0ULL) },
+    { make_floatx80_init(0x3ffd, 0xffffffffffff1c23ULL),
+      make_floatx80_init(0x3fff, 0xb504f333f9de2cadULL),
+      make_floatx80_init(0x3ffd, 0xd413cccfe778b2b4ULL) },
+    { make_floatx80_init(0x3ffe, 0x8800000000006344ULL),
+      make_floatx80_init(0x3fff, 0xb8fbaf4762fbd0a1ULL),
+      make_floatx80_init(0x3ffd, 0xe3eebd1d8bef4284ULL) },
+    { make_floatx80_init(0x3ffe, 0x9000000000005d67ULL),
+      make_floatx80_init(0x3fff, 0xbd08a39f580c668dULL),
+      make_floatx80_init(0x3ffd, 0xf4228e7d60319a34ULL) },
+    { make_floatx80_init(0x3ffe, 0x9800000000009127ULL),
+      make_floatx80_init(0x3fff, 0xc12c4cca6670e042ULL),
+      make_floatx80_init(0x3ffe, 0x82589994cce1c084ULL) },
+    { make_floatx80_init(0x3ffe, 0x9fffffffffff06f9ULL),
+      make_floatx80_init(0x3fff, 0xc5672a11550655c3ULL),
+      make_floatx80_init(0x3ffe, 0x8ace5422aa0cab86ULL) },
+    { make_floatx80_init(0x3ffe, 0xa7fffffffffff80dULL),
+      make_floatx80_init(0x3fff, 0xc9b9bd866e2f234bULL),
+      make_floatx80_init(0x3ffe, 0x93737b0cdc5e4696ULL) },
+    { make_floatx80_init(0x3ffe, 0xafffffffffff1470ULL),
+      make_floatx80_init(0x3fff, 0xce248c151f83fd69ULL),
+      make_floatx80_init(0x3ffe, 0x9c49182a3f07fad2ULL) },
+    { make_floatx80_init(0x3ffe, 0xb800000000000e0aULL),
+      make_floatx80_init(0x3fff, 0xd2a81d91f12aec5cULL),
+      make_floatx80_init(0x3ffe, 0xa5503b23e255d8b8ULL) },
+    { make_floatx80_init(0x3ffe, 0xc00000000000b7faULL),
+      make_floatx80_init(0x3fff, 0xd744fccad69dd630ULL),
+      make_floatx80_init(0x3ffe, 0xae89f995ad3bac60ULL) },
+    { make_floatx80_init(0x3ffe, 0xc800000000003aa6ULL),
+      make_floatx80_init(0x3fff, 0xdbfbb797daf25a44ULL),
+      make_floatx80_init(0x3ffe, 0xb7f76f2fb5e4b488ULL) },
+    { make_floatx80_init(0x3ffe, 0xd00000000000a6aeULL),
+      make_floatx80_init(0x3fff, 0xe0ccdeec2a954685ULL),
+      make_floatx80_init(0x3ffe, 0xc199bdd8552a8d0aULL) },
+    { make_floatx80_init(0x3ffe, 0xd800000000004165ULL),
+      make_floatx80_init(0x3fff, 0xe5b906e77c837155ULL),
+      make_floatx80_init(0x3ffe, 0xcb720dcef906e2aaULL) },
+    { make_floatx80_init(0x3ffe, 0xe00000000000582cULL),
+      make_floatx80_init(0x3fff, 0xeac0c6e7dd24713aULL),
+      make_floatx80_init(0x3ffe, 0xd5818dcfba48e274ULL) },
+    { make_floatx80_init(0x3ffe, 0xe800000000001a5dULL),
+      make_floatx80_init(0x3fff, 0xefe4b99bdcdb06ebULL),
+      make_floatx80_init(0x3ffe, 0xdfc97337b9b60dd6ULL) },
+    { make_floatx80_init(0x3ffe, 0xefffffffffffc1efULL),
+      make_floatx80_init(0x3fff, 0xf5257d152486a2faULL),
+      make_floatx80_init(0x3ffe, 0xea4afa2a490d45f4ULL) },
+    { make_floatx80_init(0x3ffe, 0xf800000000001069ULL),
+      make_floatx80_init(0x3fff, 0xfa83b2db722a0e5cULL),
+      make_floatx80_init(0x3ffe, 0xf50765b6e4541cb8ULL) },
+    { make_floatx80_init(0x3fff, 0x8000000000000000ULL),
+      make_floatx80_init(0x4000, 0x8000000000000000ULL),
+      make_floatx80_init(0x3fff, 0x8000000000000000ULL) },
 };
 
 void helper_f2xm1(CPUX86State *env)
@@ -1275,24 +1275,24 @@ struct fpatan_data {
 };
 
 static const struct fpatan_data fpatan_table[9] = {
-    { floatx80_zero,
-      floatx80_zero },
-    { make_floatx80(0x3ffb, 0xfeadd4d5617b6e33ULL),
-      make_floatx80(0xbfb9, 0xdda19d8305ddc420ULL) },
-    { make_floatx80(0x3ffc, 0xfadbafc96406eb15ULL),
-      make_floatx80(0x3fbb, 0xdb8f3debef442fccULL) },
-    { make_floatx80(0x3ffd, 0xb7b0ca0f26f78474ULL),
-      make_floatx80(0xbfbc, 0xeab9bdba460376faULL) },
-    { make_floatx80(0x3ffd, 0xed63382b0dda7b45ULL),
-      make_floatx80(0x3fbc, 0xdfc88bd978751a06ULL) },
-    { make_floatx80(0x3ffe, 0x8f005d5ef7f59f9bULL),
-      make_floatx80(0x3fbd, 0xb906bc2ccb886e90ULL) },
-    { make_floatx80(0x3ffe, 0xa4bc7d1934f70924ULL),
-      make_floatx80(0x3fbb, 0xcd43f9522bed64f8ULL) },
-    { make_floatx80(0x3ffe, 0xb8053e2bc2319e74ULL),
-      make_floatx80(0xbfbc, 0xd3496ab7bd6eef0cULL) },
-    { make_floatx80(0x3ffe, 0xc90fdaa22168c235ULL),
-      make_floatx80(0xbfbc, 0xece675d1fc8f8cbcULL) },
+    { floatx80_zero_init,
+      floatx80_zero_init },
+    { make_floatx80_init(0x3ffb, 0xfeadd4d5617b6e33ULL),
+      make_floatx80_init(0xbfb9, 0xdda19d8305ddc420ULL) },
+    { make_floatx80_init(0x3ffc, 0xfadbafc96406eb15ULL),
+      make_floatx80_init(0x3fbb, 0xdb8f3debef442fccULL) },
+    { make_floatx80_init(0x3ffd, 0xb7b0ca0f26f78474ULL),
+      make_floatx80_init(0xbfbc, 0xeab9bdba460376faULL) },
+    { make_floatx80_init(0x3ffd, 0xed63382b0dda7b45ULL),
+      make_floatx80_init(0x3fbc, 0xdfc88bd978751a06ULL) },
+    { make_floatx80_init(0x3ffe, 0x8f005d5ef7f59f9bULL),
+      make_floatx80_init(0x3fbd, 0xb906bc2ccb886e90ULL) },
+    { make_floatx80_init(0x3ffe, 0xa4bc7d1934f70924ULL),
+      make_floatx80_init(0x3fbb, 0xcd43f9522bed64f8ULL) },
+    { make_floatx80_init(0x3ffe, 0xb8053e2bc2319e74ULL),
+      make_floatx80_init(0xbfbc, 0xd3496ab7bd6eef0cULL) },
+    { make_floatx80_init(0x3ffe, 0xc90fdaa22168c235ULL),
+      make_floatx80_init(0xbfbc, 0xece675d1fc8f8cbcULL) },
 };
 
 void helper_fpatan(CPUX86State *env)
-- 
2.20.1

Re: [PATCH v3 07/16] target/i386: floatx80: avoid compound literals in static initializers

[Richard Henderson]

On 7/23/20 11:45 PM, Alex Bennée wrote:
> From: Laszlo Ersek <lersek@redhat.com>
> 
> Quoting ISO C99 6.7.8p4, "All the expressions in an initializer for an
> object that has static storage duration shall be constant expressions or
> string literals".
> 
> The compound literal produced by the make_floatx80() macro is not such a
> constant expression, per 6.6p7-9. (An implementation may accept it,
> according to 6.6p10, but is not required to.)
> 
> Therefore using "floatx80_zero" and make_floatx80() for initializing
> "f2xm1_table" and "fpatan_table" is not portable. And gcc-4.8 in RHEL-7.6
> actually chokes on them:
> 
>> target/i386/fpu_helper.c:871:5: error: initializer element is not constant
>>      { make_floatx80(0xbfff, 0x8000000000000000ULL),
>>      ^
> We've had the make_floatx80_init() macro for this purpose since commit
> 3bf7e40ab914 ("softfloat: fix for C99", 2012-03-17), so let's use that
> macro again.
> 
> Fixes: eca30647fc07
> Fixes: ff57bb7b6326
> Signed-off-by: Laszlo Ersek <lersek@redhat.com>
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
> Cc: Aurelien Jarno <aurelien@aurel32.net>
> Cc: Eduardo Habkost <ehabkost@redhat.com>
> Cc: Joseph Myers <joseph@codesourcery.com>
> Cc: Paolo Bonzini <pbonzini@redhat.com>
> Cc: Peter Maydell <peter.maydell@linaro.org>
> Cc: Richard Henderson <rth@twiddle.net>
> Link: https://lists.gnu.org/archive/html/qemu-devel/2017-08/msg06566.html
> Link: https://lists.gnu.org/archive/html/qemu-devel/2020-07/msg04714.html
> Message-Id: <20200716144251.23004-1-lersek@redhat.com>
> Message-Id: <20200722062902.24509-8-alex.bennee@linaro.org>
> ---
>  include/fpu/softfloat.h  |   1 +
>  target/i386/fpu_helper.c | 426 +++++++++++++++++++--------------------
>  2 files changed, 214 insertions(+), 213 deletions(-)

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>

r~

[PATCH v3 08/16] linux-user: don't use MAP_FIXED in pgd_find_hole_fallback

[Alex Bennée]

Plain MAP_FIXED has the undesirable behaviour of splatting exiting
maps so we don't actually achieve what we want when looking for gaps.
We should be using MAP_FIXED_NOREPLACE. As this isn't always available
we need to potentially check the returned address to see if the kernel
gave us what we asked for.

Fixes: ad592e37dfc ("linux-user: provide fallback pgd_find_hole for bare chroots")
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20200722062902.24509-9-alex.bennee@linaro.org>
---
 include/qemu/osdep.h |  3 +++
 linux-user/elfload.c | 10 ++++++----
 2 files changed, 9 insertions(+), 4 deletions(-)

diff --git a/include/qemu/osdep.h b/include/qemu/osdep.h
index 0b1298b3c91..20872e793e4 100644
--- a/include/qemu/osdep.h
+++ b/include/qemu/osdep.h
@@ -173,6 +173,9 @@ extern int daemon(int, int);
 #ifndef MAP_ANONYMOUS
 #define MAP_ANONYMOUS MAP_ANON
 #endif
+#ifndef MAP_FIXED_NOREPLACE
+#define MAP_FIXED_NOREPLACE 0
+#endif
 #ifndef ENOMEDIUM
 #define ENOMEDIUM ENODEV
 #endif
diff --git a/linux-user/elfload.c b/linux-user/elfload.c
index 7e7f642332d..fe9dfe795dd 100644
--- a/linux-user/elfload.c
+++ b/linux-user/elfload.c
@@ -2134,12 +2134,15 @@ static uintptr_t pgd_find_hole_fallback(uintptr_t guest_size, uintptr_t brk,
             /* we have run out of space */
             return -1;
         } else {
-            int flags = MAP_ANONYMOUS | MAP_PRIVATE | MAP_NORESERVE | MAP_FIXED;
+            int flags = MAP_ANONYMOUS | MAP_PRIVATE | MAP_NORESERVE |
+                MAP_FIXED_NOREPLACE;
             void * mmap_start = mmap((void *) align_start, guest_size,
                                      PROT_NONE, flags, -1, 0);
             if (mmap_start != MAP_FAILED) {
                 munmap((void *) align_start, guest_size);
-                return (uintptr_t) mmap_start + offset;
+                if (MAP_FIXED_NOREPLACE || mmap_start == (void *) align_start) {
+                    return (uintptr_t) mmap_start + offset;
+                }
             }
             base += qemu_host_page_size;
         }
@@ -2307,9 +2310,8 @@ static void pgb_reserved_va(const char *image_name, abi_ulong guest_loaddr,
     /* Widen the "image" to the entire reserved address space. */
     pgb_static(image_name, 0, reserved_va, align);
 
-#ifdef MAP_FIXED_NOREPLACE
+    /* osdep.h defines this as 0 if it's missing */
     flags |= MAP_FIXED_NOREPLACE;
-#endif
 
     /* Reserve the memory on the host. */
     assert(guest_base != 0);
-- 
2.20.1

[PATCH v3 09/16] tests/docker: fix update command due to python3 str/bytes distinction

[Alex Bennée]

Does this seem convoluted to you? It feels a little complicated to me.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200722062902.24509-10-alex.bennee@linaro.org>
---
 tests/docker/docker.py | 13 +++++++------
 1 file changed, 7 insertions(+), 6 deletions(-)

diff --git a/tests/docker/docker.py b/tests/docker/docker.py
index c9f20d8d093..356d7618f1a 100755
--- a/tests/docker/docker.py
+++ b/tests/docker/docker.py
@@ -24,7 +24,7 @@ import tempfile
 import re
 import signal
 from tarfile import TarFile, TarInfo
-from io import StringIO
+from io import StringIO, BytesIO
 from shutil import copy, rmtree
 from pwd import getpwuid
 from datetime import datetime, timedelta
@@ -541,13 +541,14 @@ class UpdateCommand(SubCommand):
 
         # Create a Docker buildfile
         df = StringIO()
-        df.write("FROM %s\n" % args.tag)
-        df.write("ADD . /\n")
-        df.seek(0)
+        df.write(u"FROM %s\n" % args.tag)
+        df.write(u"ADD . /\n")
+
+        df_bytes = BytesIO(bytes(df.getvalue(), "UTF-8"))
 
         df_tar = TarInfo(name="Dockerfile")
-        df_tar.size = len(df.buf)
-        tmp_tar.addfile(df_tar, fileobj=df)
+        df_tar.size = df_bytes.getbuffer().nbytes
+        tmp_tar.addfile(df_tar, fileobj=df_bytes)
 
         tmp_tar.close()
 
-- 
2.20.1

[PATCH v3 10/16] tests/docker: fix binfmt_misc image building

[Alex Bennée]

When we updated the arguments for docker.py we missed a bit.

Fixes: dfae6284
Fixes: dfae628459 ("docker.py/build: support -t and -f arguments")
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20200722062902.24509-11-alex.bennee@linaro.org>
---
 tests/docker/Makefile.include | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index a104e9df281..9119dff97de 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -78,7 +78,7 @@ docker-binfmt-image-debian-%: $(DOCKER_FILES_DIR)/debian-bootstrap.docker
 			DEB_ARCH=$(DEB_ARCH)					\
 			DEB_TYPE=$(DEB_TYPE) 					\
 			$(if $(DEB_URL),DEB_URL=$(DEB_URL),)			\
-			$(DOCKER_SCRIPT) build qemu/debian-$* $< 		\
+			$(DOCKER_SCRIPT) build -t qemu/debian-$* -f $< 		\
 			$(if $V,,--quiet) $(if $(NOCACHE),--no-cache) 		\
 			$(if $(NOUSER),,--add-current-user) 			\
 			$(if $(EXTRA_FILES),--extra-files $(EXTRA_FILES))	\
-- 
2.20.1

[PATCH v3 11/16] tests/docker: add support for DEB_KEYRING

[Alex Bennée]

For installing stuff from sid or ports you may need to manually
specify the location of the keyring. You can even import keys into
your personal keyring and point it there, e.g.:

  gpg --keyserver keyring.debian.org --recv-keys 84C573CD4E1AFD6C
  make docker-binfmt-image-debian-sid-hppa DEB_TYPE=sid DEB_ARCH=hppa \
      DEB_URL=http://ftp.ports.debian.org/debian-ports/ \
      EXECUTABLE=./hppa-linux-user/qemu-hppa V=1 \
      DEB_KEYRING=${HOME}/.gnupg/pubring.kbx

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20200722062902.24509-12-alex.bennee@linaro.org>
---
 tests/docker/dockerfiles/debian-bootstrap.pre | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/tests/docker/dockerfiles/debian-bootstrap.pre b/tests/docker/dockerfiles/debian-bootstrap.pre
index c164778c302..35c85f7db8a 100755
--- a/tests/docker/dockerfiles/debian-bootstrap.pre
+++ b/tests/docker/dockerfiles/debian-bootstrap.pre
@@ -79,6 +79,13 @@ else
     fi
 fi
 
+#
+# Add optional args
+#
+if [ -n "${DEB_KEYRING}" ]; then
+    DEBOOTSTRAP="${DEBOOTSTRAP} --keyring=${DEB_KEYRING}"
+fi
+
 #
 # Finally check to see if any qemu's are installed
 #
-- 
2.20.1

[PATCH v3 12/16] linux-user: fix clock_nanosleep()

[Alex Bennée]

From: Laurent Vivier <laurent@vivier.eu>

If the call is interrupted by a signal handler, it fails with error EINTR
and if "remain" is not NULL and "flags" is not TIMER_ABSTIME, it returns
the remaining unslept time in "remain".

Update linux-user to not overwrite the "remain" structure if there is no
error.

Found with "make check-tcg", linux-test fails on nanosleep test:

  TEST    linux-test on x86_64
.../tests/tcg/multiarch/linux-test.c:242: nanosleep

Reported-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200722174612.2917566-2-laurent@vivier.eu>
---
 linux-user/syscall.c | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/linux-user/syscall.c b/linux-user/syscall.c
index 1211e759c26..43a6e283961 100644
--- a/linux-user/syscall.c
+++ b/linux-user/syscall.c
@@ -11831,8 +11831,14 @@ static abi_long do_syscall1(void *cpu_env, int num, abi_long arg1,
         target_to_host_timespec(&ts, arg3);
         ret = get_errno(safe_clock_nanosleep(arg1, arg2,
                                              &ts, arg4 ? &ts : NULL));
-        if (arg4)
+        /*
+         * if the call is interrupted by a signal handler, it fails
+         * with error -TARGET_EINTR and if arg4 is not NULL and arg2 is not
+         * TIMER_ABSTIME, it returns the remaining unslept time in arg4.
+         */
+        if (ret == -TARGET_EINTR && arg4 && arg2 != TIMER_ABSTIME) {
             host_to_target_timespec(arg4, &ts);
+        }
 
 #if defined(TARGET_PPC)
         /* clock_nanosleep is odd in that it returns positive errno values.
-- 
2.20.1

Re: [PATCH v3 12/16] linux-user: fix clock_nanosleep()

[Richard Henderson]

On 7/23/20 11:45 PM, Alex Bennée wrote:
> From: Laurent Vivier <laurent@vivier.eu>
> 
> If the call is interrupted by a signal handler, it fails with error EINTR
> and if "remain" is not NULL and "flags" is not TIMER_ABSTIME, it returns
> the remaining unslept time in "remain".
> 
> Update linux-user to not overwrite the "remain" structure if there is no
> error.
> 
> Found with "make check-tcg", linux-test fails on nanosleep test:
> 
>   TEST    linux-test on x86_64
> .../tests/tcg/multiarch/linux-test.c:242: nanosleep
> 
> Reported-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
> Signed-off-by: Laurent Vivier <laurent@vivier.eu>
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> Message-Id: <20200722174612.2917566-2-laurent@vivier.eu>
> ---
>  linux-user/syscall.c | 8 +++++++-
>  1 file changed, 7 insertions(+), 1 deletion(-)

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>

r~

[PATCH v3 13/16] linux-user, ppc: fix clock_nanosleep() for linux-user-ppc

[Alex Bennée]

From: Laurent Vivier <laurent@vivier.eu>

Our safe_clock_nanosleep() returns -1 and updates errno.

We don't need to update the CRF bit in syscall.c because it will
be updated in ppc/cpu_loop.c as the return value is negative.

Signed-off-by: Laurent Vivier <laurent@vivier.eu>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200722174612.2917566-3-laurent@vivier.eu>
---
 linux-user/syscall.c | 7 -------
 1 file changed, 7 deletions(-)

diff --git a/linux-user/syscall.c b/linux-user/syscall.c
index 43a6e283961..f5c4f6b95db 100644
--- a/linux-user/syscall.c
+++ b/linux-user/syscall.c
@@ -11840,13 +11840,6 @@ static abi_long do_syscall1(void *cpu_env, int num, abi_long arg1,
             host_to_target_timespec(arg4, &ts);
         }
 
-#if defined(TARGET_PPC)
-        /* clock_nanosleep is odd in that it returns positive errno values.
-         * On PPC, CR0 bit 3 should be set in such a situation. */
-        if (ret && ret != -TARGET_ERESTARTSYS) {
-            ((CPUPPCState *)cpu_env)->crf[0] |= 1;
-        }
-#endif
         return ret;
     }
 #endif
-- 
2.20.1

Re: [PATCH v3 13/16] linux-user,ppc: fix clock_nanosleep() for linux-user-ppc

[Richard Henderson]

On 7/23/20 11:45 PM, Alex Bennée wrote:
> From: Laurent Vivier <laurent@vivier.eu>
> 
> Our safe_clock_nanosleep() returns -1 and updates errno.
> 
> We don't need to update the CRF bit in syscall.c because it will
> be updated in ppc/cpu_loop.c as the return value is negative.
> 
> Signed-off-by: Laurent Vivier <laurent@vivier.eu>
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> Message-Id: <20200722174612.2917566-3-laurent@vivier.eu>
> ---
>  linux-user/syscall.c | 7 -------
>  1 file changed, 7 deletions(-)

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>

r~

[PATCH v3 14/16] python/qemu: Cleanup changes to ConsoleSocket

[Alex Bennée]

From: Robert Foley <robert.foley@linaro.org>

The changes to console_socket.py and machine.py are to
cleanup for pylint and flake8.

Signed-off-by: Robert Foley <robert.foley@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200717203041.9867-2-robert.foley@linaro.org>
---
 python/qemu/console_socket.py | 57 ++++++++++++++++++-----------------
 python/qemu/machine.py        |  7 +++--
 python/qemu/pylintrc          |  2 +-
 3 files changed, 34 insertions(+), 32 deletions(-)

diff --git a/python/qemu/console_socket.py b/python/qemu/console_socket.py
index 830cb7c6282..09986bc2152 100644
--- a/python/qemu/console_socket.py
+++ b/python/qemu/console_socket.py
@@ -1,12 +1,9 @@
-#!/usr/bin/env python3
-#
-# This python module implements a ConsoleSocket object which is
-# designed always drain the socket itself, and place
-# the bytes into a in memory buffer for later processing.
-#
-# Optionally a file path can be passed in and we will also
-# dump the characters to this file for debug.
-#
+"""
+QEMU Console Socket Module:
+
+This python module implements a ConsoleSocket object,
+which can drain a socket and optionally dump the bytes to file.
+"""
 # Copyright 2020 Linaro
 #
 # Authors:
@@ -15,20 +12,27 @@
 # This code is licensed under the GPL version 2 or later.  See
 # the COPYING file in the top-level directory.
 #
+
 import asyncore
 import socket
 import threading
-import io
-import os
-import sys
 from collections import deque
 import time
-import traceback
+
 
 class ConsoleSocket(asyncore.dispatcher):
+    """
+    ConsoleSocket represents a socket attached to a char device.
 
+    Drains the socket and places the bytes into an in memory buffer
+    for later processing.
+
+    Optionally a file path can be passed in and we will also
+    dump the characters to this file for debugging purposes.
+    """
     def __init__(self, address, file=None):
         self._recv_timeout_sec = 300
+        self._sleep_time = 0.5
         self._buffer = deque()
         self._asyncore_thread = None
         self._sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
@@ -70,31 +74,28 @@ class ConsoleSocket(asyncore.dispatcher):
 
     def handle_read(self):
         """process arriving characters into in memory _buffer"""
-        try:
-            data = asyncore.dispatcher.recv(self, 1)
-            # latin1 is needed since there are some chars
-            # we are receiving that cannot be encoded to utf-8
-            # such as 0xe2, 0x80, 0xA6.
-            string = data.decode("latin1")
-        except:
-            print("Exception seen.")
-            traceback.print_exc()
-            return
+        data = asyncore.dispatcher.recv(self, 1)
+        # latin1 is needed since there are some chars
+        # we are receiving that cannot be encoded to utf-8
+        # such as 0xe2, 0x80, 0xA6.
+        string = data.decode("latin1")
         if self._logfile:
             self._logfile.write("{}".format(string))
             self._logfile.flush()
         for c in string:
             self._buffer.extend(c)
 
-    def recv(self, n=1, sleep_delay_s=0.1):
-        """Return chars from in memory buffer"""
+    def recv(self, buffer_size=1):
+        """Return chars from in memory buffer.
+           Maintains the same API as socket.socket.recv.
+        """
         start_time = time.time()
-        while len(self._buffer) < n:
-            time.sleep(sleep_delay_s)
+        while len(self._buffer) < buffer_size:
+            time.sleep(self._sleep_time)
             elapsed_sec = time.time() - start_time
             if elapsed_sec > self._recv_timeout_sec:
                 raise socket.timeout
-        chars = ''.join([self._buffer.popleft() for i in range(n)])
+        chars = ''.join([self._buffer.popleft() for i in range(buffer_size)])
         # We choose to use latin1 to remain consistent with
         # handle_read() and give back the same data as the user would
         # receive if they were reading directly from the
diff --git a/python/qemu/machine.py b/python/qemu/machine.py
index 80c4d4a8b6e..9956360a792 100644
--- a/python/qemu/machine.py
+++ b/python/qemu/machine.py
@@ -27,7 +27,7 @@ import socket
 import tempfile
 from typing import Optional, Type
 from types import TracebackType
-from qemu.console_socket import ConsoleSocket
+from . import console_socket
 
 from . import qmp
 
@@ -674,8 +674,9 @@ class QEMUMachine:
         """
         if self._console_socket is None:
             if self._drain_console:
-                self._console_socket = ConsoleSocket(self._console_address,
-                                                    file=self._console_log_path)
+                self._console_socket = console_socket.ConsoleSocket(
+                    self._console_address,
+                    file=self._console_log_path)
             else:
                 self._console_socket = socket.socket(socket.AF_UNIX,
                                                      socket.SOCK_STREAM)
diff --git a/python/qemu/pylintrc b/python/qemu/pylintrc
index 5d6ae7367d8..3f69205000d 100644
--- a/python/qemu/pylintrc
+++ b/python/qemu/pylintrc
@@ -33,7 +33,7 @@ good-names=i,
            Run,
            _,
            fd,
-
+           c,
 [VARIABLES]
 
 [STRING]
-- 
2.20.1

Re: [PATCH v3 14/16] python/qemu: Cleanup changes to ConsoleSocket

[Philippe Mathieu-Daudé]

+John who requested the changes.

On 7/24/20 8:45 AM, Alex Bennée wrote:
> From: Robert Foley <robert.foley@linaro.org>
> 
> The changes to console_socket.py and machine.py are to
> cleanup for pylint and flake8.
> 
> Signed-off-by: Robert Foley <robert.foley@linaro.org>
> Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> Message-Id: <20200717203041.9867-2-robert.foley@linaro.org>
> ---
>  python/qemu/console_socket.py | 57 ++++++++++++++++++-----------------
>  python/qemu/machine.py        |  7 +++--
>  python/qemu/pylintrc          |  2 +-
>  3 files changed, 34 insertions(+), 32 deletions(-)
> 
> diff --git a/python/qemu/console_socket.py b/python/qemu/console_socket.py
> index 830cb7c6282..09986bc2152 100644
> --- a/python/qemu/console_socket.py
> +++ b/python/qemu/console_socket.py
> @@ -1,12 +1,9 @@
> -#!/usr/bin/env python3
> -#
> -# This python module implements a ConsoleSocket object which is
> -# designed always drain the socket itself, and place
> -# the bytes into a in memory buffer for later processing.
> -#
> -# Optionally a file path can be passed in and we will also
> -# dump the characters to this file for debug.
> -#
> +"""
> +QEMU Console Socket Module:
> +
> +This python module implements a ConsoleSocket object,
> +which can drain a socket and optionally dump the bytes to file.
> +"""
>  # Copyright 2020 Linaro
>  #
>  # Authors:
> @@ -15,20 +12,27 @@
>  # This code is licensed under the GPL version 2 or later.  See
>  # the COPYING file in the top-level directory.
>  #
> +
>  import asyncore
>  import socket
>  import threading
> -import io
> -import os
> -import sys
>  from collections import deque
>  import time
> -import traceback
> +
>  
>  class ConsoleSocket(asyncore.dispatcher):
> +    """
> +    ConsoleSocket represents a socket attached to a char device.
>  
> +    Drains the socket and places the bytes into an in memory buffer
> +    for later processing.
> +
> +    Optionally a file path can be passed in and we will also
> +    dump the characters to this file for debugging purposes.
> +    """
>      def __init__(self, address, file=None):
>          self._recv_timeout_sec = 300
> +        self._sleep_time = 0.5
>          self._buffer = deque()
>          self._asyncore_thread = None
>          self._sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
> @@ -70,31 +74,28 @@ class ConsoleSocket(asyncore.dispatcher):
>  
>      def handle_read(self):
>          """process arriving characters into in memory _buffer"""
> -        try:
> -            data = asyncore.dispatcher.recv(self, 1)
> -            # latin1 is needed since there are some chars
> -            # we are receiving that cannot be encoded to utf-8
> -            # such as 0xe2, 0x80, 0xA6.
> -            string = data.decode("latin1")
> -        except:
> -            print("Exception seen.")
> -            traceback.print_exc()
> -            return
> +        data = asyncore.dispatcher.recv(self, 1)

A bit more than the commit description, but I don't mind.

Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>

> +        # latin1 is needed since there are some chars
> +        # we are receiving that cannot be encoded to utf-8
> +        # such as 0xe2, 0x80, 0xA6.
> +        string = data.decode("latin1")
>          if self._logfile:
>              self._logfile.write("{}".format(string))
>              self._logfile.flush()
>          for c in string:
>              self._buffer.extend(c)
>  
> -    def recv(self, n=1, sleep_delay_s=0.1):
> -        """Return chars from in memory buffer"""
> +    def recv(self, buffer_size=1):
> +        """Return chars from in memory buffer.
> +           Maintains the same API as socket.socket.recv.
> +        """
>          start_time = time.time()
> -        while len(self._buffer) < n:
> -            time.sleep(sleep_delay_s)
> +        while len(self._buffer) < buffer_size:
> +            time.sleep(self._sleep_time)
>              elapsed_sec = time.time() - start_time
>              if elapsed_sec > self._recv_timeout_sec:
>                  raise socket.timeout
> -        chars = ''.join([self._buffer.popleft() for i in range(n)])
> +        chars = ''.join([self._buffer.popleft() for i in range(buffer_size)])
>          # We choose to use latin1 to remain consistent with
>          # handle_read() and give back the same data as the user would
>          # receive if they were reading directly from the
> diff --git a/python/qemu/machine.py b/python/qemu/machine.py
> index 80c4d4a8b6e..9956360a792 100644
> --- a/python/qemu/machine.py
> +++ b/python/qemu/machine.py
> @@ -27,7 +27,7 @@ import socket
>  import tempfile
>  from typing import Optional, Type
>  from types import TracebackType
> -from qemu.console_socket import ConsoleSocket
> +from . import console_socket
>  
>  from . import qmp
>  
> @@ -674,8 +674,9 @@ class QEMUMachine:
>          """
>          if self._console_socket is None:
>              if self._drain_console:
> -                self._console_socket = ConsoleSocket(self._console_address,
> -                                                    file=self._console_log_path)
> +                self._console_socket = console_socket.ConsoleSocket(
> +                    self._console_address,
> +                    file=self._console_log_path)
>              else:
>                  self._console_socket = socket.socket(socket.AF_UNIX,
>                                                       socket.SOCK_STREAM)
> diff --git a/python/qemu/pylintrc b/python/qemu/pylintrc
> index 5d6ae7367d8..3f69205000d 100644
> --- a/python/qemu/pylintrc
> +++ b/python/qemu/pylintrc
> @@ -33,7 +33,7 @@ good-names=i,
>             Run,
>             _,
>             fd,
> -
> +           c,
>  [VARIABLES]
>  
>  [STRING]
> 

[PATCH v3 15/16] python/qemu: Change ConsoleSocket to optionally drain socket.

[Alex Bennée]

From: Robert Foley <robert.foley@linaro.org>

The primary purpose of this change is to clean up
machine.py's console_socket property to return a single type,
a ConsoleSocket.

ConsoleSocket now derives from a socket, which means that
in the default case (of not draining), machine.py
will see the same behavior as it did prior to ConsoleSocket.

Signed-off-by: Robert Foley <robert.foley@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200717203041.9867-3-robert.foley@linaro.org>
---
 python/qemu/console_socket.py | 92 +++++++++++++++++++++--------------
 python/qemu/machine.py        | 13 ++---
 2 files changed, 59 insertions(+), 46 deletions(-)

diff --git a/python/qemu/console_socket.py b/python/qemu/console_socket.py
index 09986bc2152..70869fbbdc4 100644
--- a/python/qemu/console_socket.py
+++ b/python/qemu/console_socket.py
@@ -13,68 +13,75 @@ which can drain a socket and optionally dump the bytes to file.
 # the COPYING file in the top-level directory.
 #
 
-import asyncore
 import socket
 import threading
 from collections import deque
 import time
 
 
-class ConsoleSocket(asyncore.dispatcher):
+class ConsoleSocket(socket.socket):
     """
     ConsoleSocket represents a socket attached to a char device.
 
-    Drains the socket and places the bytes into an in memory buffer
-    for later processing.
+    Optionally (if drain==True), drains the socket and places the bytes
+    into an in memory buffer for later processing.
 
     Optionally a file path can be passed in and we will also
     dump the characters to this file for debugging purposes.
     """
-    def __init__(self, address, file=None):
+    def __init__(self, address, file=None, drain=False):
         self._recv_timeout_sec = 300
         self._sleep_time = 0.5
         self._buffer = deque()
-        self._asyncore_thread = None
-        self._sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
-        self._sock.connect(address)
+        socket.socket.__init__(self, socket.AF_UNIX, socket.SOCK_STREAM)
+        self.connect(address)
         self._logfile = None
         if file:
             self._logfile = open(file, "w")
-        asyncore.dispatcher.__init__(self, sock=self._sock)
         self._open = True
-        self._thread_start()
+        if drain:
+            self._drain_thread = self._thread_start()
+        else:
+            self._drain_thread = None
 
-    def _thread_start(self):
-        """Kick off a thread to wait on the asyncore.loop"""
-        if self._asyncore_thread is not None:
-            return
-        self._asyncore_thread = threading.Thread(target=asyncore.loop,
-                                                 kwargs={'timeout':1})
-        self._asyncore_thread.daemon = True
-        self._asyncore_thread.start()
+    def _drain_fn(self):
+        """Drains the socket and runs while the socket is open."""
+        while self._open:
+            try:
+                self._drain_socket()
+            except socket.timeout:
+                # The socket is expected to timeout since we set a
+                # short timeout to allow the thread to exit when
+                # self._open is set to False.
+                time.sleep(self._sleep_time)
 
-    def handle_close(self):
-        """redirect close to base class"""
-        # Call the base class close, but not self.close() since
-        # handle_close() occurs in the context of the thread which
-        # self.close() attempts to join.
-        asyncore.dispatcher.close(self)
+    def _thread_start(self):
+        """Kick off a thread to drain the socket."""
+        # Configure socket to not block and timeout.
+        # This allows our drain thread to not block
+        # on recieve and exit smoothly.
+        socket.socket.setblocking(self, False)
+        socket.socket.settimeout(self, 1)
+        drain_thread = threading.Thread(target=self._drain_fn)
+        drain_thread.daemon = True
+        drain_thread.start()
+        return drain_thread
 
     def close(self):
         """Close the base object and wait for the thread to terminate"""
         if self._open:
             self._open = False
-            asyncore.dispatcher.close(self)
-            if self._asyncore_thread is not None:
-                thread, self._asyncore_thread = self._asyncore_thread, None
+            if self._drain_thread is not None:
+                thread, self._drain_thread = self._drain_thread, None
                 thread.join()
+            socket.socket.close(self)
             if self._logfile:
                 self._logfile.close()
                 self._logfile = None
 
-    def handle_read(self):
+    def _drain_socket(self):
         """process arriving characters into in memory _buffer"""
-        data = asyncore.dispatcher.recv(self, 1)
+        data = socket.socket.recv(self, 1)
         # latin1 is needed since there are some chars
         # we are receiving that cannot be encoded to utf-8
         # such as 0xe2, 0x80, 0xA6.
@@ -85,27 +92,38 @@ class ConsoleSocket(asyncore.dispatcher):
         for c in string:
             self._buffer.extend(c)
 
-    def recv(self, buffer_size=1):
+    def recv(self, bufsize=1):
         """Return chars from in memory buffer.
            Maintains the same API as socket.socket.recv.
         """
+        if self._drain_thread is None:
+            # Not buffering the socket, pass thru to socket.
+            return socket.socket.recv(self, bufsize)
         start_time = time.time()
-        while len(self._buffer) < buffer_size:
+        while len(self._buffer) < bufsize:
             time.sleep(self._sleep_time)
             elapsed_sec = time.time() - start_time
             if elapsed_sec > self._recv_timeout_sec:
                 raise socket.timeout
-        chars = ''.join([self._buffer.popleft() for i in range(buffer_size)])
+        chars = ''.join([self._buffer.popleft() for i in range(bufsize)])
         # We choose to use latin1 to remain consistent with
         # handle_read() and give back the same data as the user would
         # receive if they were reading directly from the
         # socket w/o our intervention.
         return chars.encode("latin1")
 
-    def set_blocking(self):
-        """Maintain compatibility with socket API"""
-        pass
+    def setblocking(self, value):
+        """When not draining we pass thru to the socket,
+           since when draining we control socket blocking.
+        """
+        if self._drain_thread is None:
+            socket.socket.setblocking(self, value)
 
     def settimeout(self, seconds):
-        """Set current timeout on recv"""
-        self._recv_timeout_sec = seconds
+        """When not draining we pass thru to the socket,
+           since when draining we control the timeout.
+        """
+        if seconds is not None:
+            self._recv_timeout_sec = seconds
+        if self._drain_thread is None:
+            socket.socket.settimeout(self, seconds)
diff --git a/python/qemu/machine.py b/python/qemu/machine.py
index 9956360a792..c5f777b9e72 100644
--- a/python/qemu/machine.py
+++ b/python/qemu/machine.py
@@ -23,7 +23,6 @@ import os
 import subprocess
 import shutil
 import signal
-import socket
 import tempfile
 from typing import Optional, Type
 from types import TracebackType
@@ -673,12 +672,8 @@ class QEMUMachine:
         Returns a socket connected to the console
         """
         if self._console_socket is None:
-            if self._drain_console:
-                self._console_socket = console_socket.ConsoleSocket(
-                    self._console_address,
-                    file=self._console_log_path)
-            else:
-                self._console_socket = socket.socket(socket.AF_UNIX,
-                                                     socket.SOCK_STREAM)
-                self._console_socket.connect(self._console_address)
+            self._console_socket = console_socket.ConsoleSocket(
+                self._console_address,
+                file=self._console_log_path,
+                drain=self._drain_console)
         return self._console_socket
-- 
2.20.1

[PATCH v3 16/16] tests/vm: add shutdown timeout in basevm.py

[Alex Bennée]

From: Robert Foley <robert.foley@linaro.org>

We are adding the shutdown timeout to solve an issue
we now see where the aarch64 VMs timeout on shutdown
under TCG.

There is a new 3 second timeout in machine.py,
which we override in basevm.py when shutting down.

Signed-off-by: Robert Foley <robert.foley@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200717203041.9867-4-robert.foley@linaro.org>
---
 tests/vm/basevm.py | 15 ++++++++++-----
 1 file changed, 10 insertions(+), 5 deletions(-)

diff --git a/tests/vm/basevm.py b/tests/vm/basevm.py
index 7acb48b8761..3fac20e929a 100644
--- a/tests/vm/basevm.py
+++ b/tests/vm/basevm.py
@@ -80,6 +80,8 @@ class BaseVM(object):
     arch = "#arch"
     # command to halt the guest, can be overridden by subclasses
     poweroff = "poweroff"
+    # Time to wait for shutdown to finish.
+    shutdown_timeout_default = 30
     # enable IPv6 networking
     ipv6 = True
     # This is the timeout on the wait for console bytes.
@@ -87,7 +89,7 @@ class BaseVM(object):
     # Scale up some timeouts under TCG.
     # 4 is arbitrary, but greater than 2,
     # since we found we need to wait more than twice as long.
-    tcg_ssh_timeout_multiplier = 4
+    tcg_timeout_multiplier = 4
     def __init__(self, args, config=None):
         self._guest = None
         self._genisoimage = args.genisoimage
@@ -141,9 +143,12 @@ class BaseVM(object):
         if args.jobs and args.jobs > 1:
             self._args += ["-smp", "%d" % args.jobs]
         if kvm_available(self.arch):
+            self._shutdown_timeout = self.shutdown_timeout_default
             self._args += ["-enable-kvm"]
         else:
             logging.info("KVM not available, not using -enable-kvm")
+            self._shutdown_timeout = \
+                self.shutdown_timeout_default * self.tcg_timeout_multiplier
         self._data_args = []
 
         if self._config['qemu_args'] != None:
@@ -423,7 +428,7 @@ class BaseVM(object):
     def wait_ssh(self, wait_root=False, seconds=300, cmd="exit 0"):
         # Allow more time for VM to boot under TCG.
         if not kvm_available(self.arch):
-            seconds *= self.tcg_ssh_timeout_multiplier
+            seconds *= self.tcg_timeout_multiplier
         starttime = datetime.datetime.now()
         endtime = starttime + datetime.timedelta(seconds=seconds)
         cmd_success = False
@@ -441,14 +446,14 @@ class BaseVM(object):
             raise Exception("Timeout while waiting for guest ssh")
 
     def shutdown(self):
-        self._guest.shutdown()
+        self._guest.shutdown(timeout=self._shutdown_timeout)
 
     def wait(self):
-        self._guest.wait()
+        self._guest.wait(timeout=self._shutdown_timeout)
 
     def graceful_shutdown(self):
         self.ssh_root(self.poweroff)
-        self._guest.wait()
+        self._guest.wait(timeout=self._shutdown_timeout)
 
     def qmp(self, *args, **kwargs):
         return self._guest.qmp(*args, **kwargs)
-- 
2.20.1