[Qemu-devel] [RFC 0/2] Improve qemu-thread support on macOS

[Qemu-devel] [RFC 0/2] Improve qemu-thread support on macOS

[Roman Bolshakov]

Hello,

I've hit a case where QEMU hangs not responding to anything except
SIGKILL. It turned out to be a SIGSEGV in vCPU thread that was lost by
masking all signals.

By blocking too many signals QEMU relies on undefined behaviour that
seems to work on Linux. It's documented in POSIX reference and
sigprocmask(2). Indeed signalfd(2) on Linux notes that it can't be used
to receive SIGSEGV and SIGFPE.

It's not clear what do with SIGBUS on macOS. We can't blindly unblock it
as it's used for memory preallocation.

Also the RFC adds support for thread naming on macOS. Some threads
(signalfd_compat and rcu_call) are created before debug-threads=on is
parsed and don't get their names though.

Thank you,
Roman

Roman Bolshakov (2):
  util: Implement debug-threads for macOS
  qemu-thread: Don't block SEGV, ILL and FPE

 configure                | 32 ++++++++++++++++++++++++++------
 qemu-options.hx          |  4 ++--
 util/qemu-thread-posix.c | 11 ++++++++++-
 3 files changed, 38 insertions(+), 9 deletions(-)

-- 
2.19.1

[Qemu-devel] [RFC 1/2] util: Implement debug-threads for macOS

[Roman Bolshakov]

macOS provides pthread_setname_np that doesn't have thread id argument.

Signed-off-by: Roman Bolshakov <r.bolshakov@yadro.com>
---
 configure                | 32 ++++++++++++++++++++++++++------
 qemu-options.hx          |  4 ++--
 util/qemu-thread-posix.c |  6 +++++-
 3 files changed, 33 insertions(+), 9 deletions(-)

diff --git a/configure b/configure
index 224d3071ac..99dc073e53 100755
--- a/configure
+++ b/configure
@@ -3715,8 +3715,8 @@ if test "$mingw32" != yes -a "$pthread" = no; then
       "Make sure to have the pthread libs and headers installed."
 fi
 
-# check for pthread_setname_np
-pthread_setname_np=no
+# check for pthread_setname_np with thread id
+pthread_setname_np_w_tid=no
 cat > $TMPC << EOF
 #include <pthread.h>
 
@@ -3730,7 +3730,24 @@ int main(void)
 }
 EOF
 if compile_prog "" "$pthread_lib" ; then
-  pthread_setname_np=yes
+  pthread_setname_np_w_tid=yes
+fi
+
+# check for pthread_setname_np without thread id
+pthread_setname_np_wo_tid=no
+cat > $TMPC << EOF
+#include <pthread.h>
+
+static void *f(void *p) { pthread_setname_np("QEMU"); }
+int main(void)
+{
+    pthread_t thread;
+    pthread_create(&thread, 0, f, 0);
+    return 0;
+}
+EOF
+if compile_prog "" "$pthread_lib" ; then
+  pthread_setname_np_wo_tid=yes
 fi
 
 ##########################################
@@ -6883,11 +6900,14 @@ fi
 # Hold two types of flag:
 #   CONFIG_THREAD_SETNAME_BYTHREAD  - we've got a way of setting the name on
 #                                     a thread we have a handle to
-#   CONFIG_PTHREAD_SETNAME_NP       - A way of doing it on a particular
+#   CONFIG_PTHREAD_SETNAME_NP_W_TID - A way of doing it on a particular
 #                                     platform
-if test "$pthread_setname_np" = "yes" ; then
+if test "$pthread_setname_np_w_tid" = "yes" ; then
   echo "CONFIG_THREAD_SETNAME_BYTHREAD=y" >> $config_host_mak
-  echo "CONFIG_PTHREAD_SETNAME_NP=y" >> $config_host_mak
+  echo "CONFIG_PTHREAD_SETNAME_NP_W_TID=y" >> $config_host_mak
+elif test "$pthread_setname_np_wo_tid" = "yes" ; then
+  echo "CONFIG_THREAD_SETNAME_BYTHREAD=y" >> $config_host_mak
+  echo "CONFIG_PTHREAD_SETNAME_NP_WO_TID=y" >> $config_host_mak
 fi
 
 if test "$vxhs" = "yes" ; then
diff --git a/qemu-options.hx b/qemu-options.hx
index df42116ecc..d4f3564b78 100644
--- a/qemu-options.hx
+++ b/qemu-options.hx
@@ -538,8 +538,8 @@ ETEXI
 DEF("name", HAS_ARG, QEMU_OPTION_name,
     "-name string1[,process=string2][,debug-threads=on|off]\n"
     "                set the name of the guest\n"
-    "                string1 sets the window title and string2 the process name (on Linux)\n"
-    "                When debug-threads is enabled, individual threads are given a separate name (on Linux)\n"
+    "                string1 sets the window title and string2 the process name\n"
+    "                When debug-threads is enabled, individual threads are given a separate name\n"
     "                NOTE: The thread names are for debugging and not a stable API.\n",
     QEMU_ARCH_ALL)
 STEXI
diff --git a/util/qemu-thread-posix.c b/util/qemu-thread-posix.c
index 865e476df5..c6934bd22c 100644
--- a/util/qemu-thread-posix.c
+++ b/util/qemu-thread-posix.c
@@ -484,12 +484,16 @@ static void *qemu_thread_start(void *args)
     void *arg = qemu_thread_args->arg;
     void *r;
 
-#ifdef CONFIG_PTHREAD_SETNAME_NP
+#ifdef CONFIG_THREAD_SETNAME_BYTHREAD
     /* Attempt to set the threads name; note that this is for debug, so
      * we're not going to fail if we can't set it.
      */
     if (name_threads && qemu_thread_args->name) {
+# if defined(CONFIG_PTHREAD_SETNAME_NP_W_TID)
         pthread_setname_np(pthread_self(), qemu_thread_args->name);
+# elif defined(CONFIG_PTHREAD_SETNAME_NP_WO_TID)
+        pthread_setname_np(qemu_thread_args->name);
+# endif
     }
 #endif
     g_free(qemu_thread_args->name);
-- 
2.19.1

Re: [Qemu-devel] [RFC 1/2] util: Implement debug-threads for macOS

[Daniel P. Berrangé]

On Mon, Dec 17, 2018 at 11:26:01PM +0300, Roman Bolshakov wrote:
> macOS provides pthread_setname_np that doesn't have thread id argument.
> 
> Signed-off-by: Roman Bolshakov <r.bolshakov@yadro.com>
> ---
>  configure                | 32 ++++++++++++++++++++++++++------
>  qemu-options.hx          |  4 ++--
>  util/qemu-thread-posix.c |  6 +++++-
>  3 files changed, 33 insertions(+), 9 deletions(-)

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>


Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

[Qemu-devel] [RFC 2/2] qemu-thread: Don't block SEGV, ILL and FPE

[Roman Bolshakov]

If any of these signals happen on macOS, they are not delivered to other
threads and signalfd_compat receives nothing. Indeed, POSIX reference
and sigprocmask(2) note that an attempt to block the signals results in
undefined behaviour. SEGV and FPE can't also be received by signalfd(2)
on Linux.

An ability to retrieve SIGBUS via signalfd(2) is used by QEMU for
memory preallocation therefore we can't unblock it without consequences.
But it's important to leave a remark that the signal is lost on macOS.

Signed-off-by: Roman Bolshakov <r.bolshakov@yadro.com>
---
 util/qemu-thread-posix.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/util/qemu-thread-posix.c b/util/qemu-thread-posix.c
index c6934bd22c..1bf5e65dea 100644
--- a/util/qemu-thread-posix.c
+++ b/util/qemu-thread-posix.c
@@ -524,6 +524,11 @@ void qemu_thread_create(QemuThread *thread, const char *name,
 
     /* Leave signal handling to the iothread.  */
     sigfillset(&set);
+    /* Blocking the signals can result in undefined behaviour. */
+    sigdelset(&set, SIGSEGV);
+    sigdelset(&set, SIGFPE);
+    sigdelset(&set, SIGILL);
+    /* TODO avoid SIGBUS loss on macOS */
     pthread_sigmask(SIG_SETMASK, &set, &oldset);
 
     qemu_thread_args = g_new0(QemuThreadArgs, 1);
-- 
2.19.1

Re: [Qemu-devel] [RFC 2/2] qemu-thread: Don't block SEGV, ILL and FPE

[Daniel P. Berrangé]

On Mon, Dec 17, 2018 at 11:26:02PM +0300, Roman Bolshakov wrote:
> If any of these signals happen on macOS, they are not delivered to other
> threads and signalfd_compat receives nothing. Indeed, POSIX reference
> and sigprocmask(2) note that an attempt to block the signals results in
> undefined behaviour. SEGV and FPE can't also be received by signalfd(2)
> on Linux.
>
> An ability to retrieve SIGBUS via signalfd(2) is used by QEMU for
> memory preallocation therefore we can't unblock it without consequences.
> But it's important to leave a remark that the signal is lost on macOS.


Specifically Linux manpage says

   "If SIGBUS, SIGFPE, SIGILL, or SIGSEGV are generated while they
    are blocked, the result is undefined, unless the signal was 
    generated by kill(2), sigqueue(3), or raise(3)."

It appears as if Linux will deliver those signals to a different
thread where they're not blocked, but macOS will simply discard
the signals. Linux behaviour is more helpful, but both are
compliant since behaviour is explicitly undefined.

So really we should not have code that relies on being able to
block any of BUS, FPE, ILL or SEGV.

Allowing FPE, ILL & SEGV looks trivial since nothing in QEMU
tries to handle them, so it'll just trigger the OS default
signal handler which will abort.

As noted code is seeming to rely on catching BUS which is a
problem and I'm not clear how, or even if, we can remove
that and thus unblock the signals.

Copying Paolo, since this unconditional blocking of all signals
was added in

  commit 55541c8afc1a2d75de890c6ee858769d7d605526
  Author: Paolo Bonzini <pbonzini@redhat.com>
  Date:   Thu Jun 3 15:20:32 2010 +0200

    make qemu_thread_create block all signals
    
    All signals will thus be routed through the IO thread.
    
    Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
    Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>


> 
> Signed-off-by: Roman Bolshakov <r.bolshakov@yadro.com>
> ---
>  util/qemu-thread-posix.c | 5 +++++
>  1 file changed, 5 insertions(+)
> 
> diff --git a/util/qemu-thread-posix.c b/util/qemu-thread-posix.c
> index c6934bd22c..1bf5e65dea 100644
> --- a/util/qemu-thread-posix.c
> +++ b/util/qemu-thread-posix.c
> @@ -524,6 +524,11 @@ void qemu_thread_create(QemuThread *thread, const char *name,
>  
>      /* Leave signal handling to the iothread.  */
>      sigfillset(&set);
> +    /* Blocking the signals can result in undefined behaviour. */
> +    sigdelset(&set, SIGSEGV);
> +    sigdelset(&set, SIGFPE);
> +    sigdelset(&set, SIGILL);
> +    /* TODO avoid SIGBUS loss on macOS */
>      pthread_sigmask(SIG_SETMASK, &set, &oldset);

Regardless of what we do with SIGBUS, this addition looks fine to me
on its own so

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>


Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [Qemu-devel] [RFC 0/2] Improve qemu-thread support on macOS

[Peter Maydell]

On Mon, 17 Dec 2018 at 20:46, Roman Bolshakov <r.bolshakov@yadro.com> wrote:
>
> Hello,
>
> I've hit a case where QEMU hangs not responding to anything except
> SIGKILL. It turned out to be a SIGSEGV in vCPU thread that was lost by
> masking all signals.
>
> By blocking too many signals QEMU relies on undefined behaviour that
> seems to work on Linux. It's documented in POSIX reference and
> sigprocmask(2). Indeed signalfd(2) on Linux notes that it can't be used
> to receive SIGSEGV and SIGFPE.
>
> It's not clear what do with SIGBUS on macOS. We can't blindly unblock it
> as it's used for memory preallocation.
>
> Also the RFC adds support for thread naming on macOS. Some threads
> (signalfd_compat and rcu_call) are created before debug-threads=on is
> parsed and don't get their names though.

Applied to master, thanks.

-- PMM