[PATCH v2 0/2] add error handling in multifd_tls_handshake_thread

[PATCH v2 0/2] add error handling in multifd_tls_handshake_thread

[Hao Wang]

If any error happens during multifd send thread creating (e.g. channel broke
because new domain is destroyed by the dst), multifd_tls_handshake_thread
may exit silently, leaving main migration thread hanging (ram_save_setup ->
multifd_send_sync_main -> qemu_sem_wait(&p->sem_sync)).
Fix that by adding error handling in multifd_tls_handshake_thread.

Hao Wang (2):
  migration/tls: fix inverted semantics in multifd_channel_connect
  migration/tls: add error handling in multifd_tls_handshake_thread

 migration/multifd.c | 21 +++++++++++++++------
 1 file changed, 15 insertions(+), 6 deletions(-)

-- 
2.23.0

[PATCH v2 1/2] migration/tls: fix inverted semantics in multifd_channel_connect

[Hao Wang]

Function multifd_channel_connect() return "true" to indicate failure,
which is rather confusing. Fix that.

Signed-off-by: Hao Wang <wanghao232@huawei.com>
---
 migration/multifd.c | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/migration/multifd.c b/migration/multifd.c
index 1a1e589064..2a1ea85ade 100644
--- a/migration/multifd.c
+++ b/migration/multifd.c
@@ -798,9 +798,9 @@ static bool multifd_channel_connect(MultiFDSendParams *p,
                  * function after the TLS handshake,
                  * so we mustn't call multifd_send_thread until then
                  */
-                return false;
-            } else {
                 return true;
+            } else {
+                return false;
             }
         } else {
             /* update for tls qio channel */
@@ -808,10 +808,10 @@ static bool multifd_channel_connect(MultiFDSendParams *p,
             qemu_thread_create(&p->thread, p->name, multifd_send_thread, p,
                                    QEMU_THREAD_JOINABLE);
        }
-       return false;
+       return true;
     }
 
-    return true;
+    return false;
 }
 
 static void multifd_new_send_channel_cleanup(MultiFDSendParams *p,
@@ -844,7 +844,7 @@ static void multifd_new_send_channel_async(QIOTask *task, gpointer opaque)
         p->c = QIO_CHANNEL(sioc);
         qio_channel_set_delay(p->c, false);
         p->running = true;
-        if (multifd_channel_connect(p, sioc, local_err)) {
+        if (!multifd_channel_connect(p, sioc, local_err)) {
             goto cleanup;
         }
         return;
-- 
2.23.0

Re: [PATCH v2 1/2] migration/tls: fix inverted semantics in multifd_channel_connect

[Daniel P. Berrangé]

On Tue, Feb 09, 2021 at 06:42:36PM +0800, Hao Wang wrote:
> Function multifd_channel_connect() return "true" to indicate failure,
> which is rather confusing. Fix that.
> 
> Signed-off-by: Hao Wang <wanghao232@huawei.com>
> ---
>  migration/multifd.c | 10 +++++-----
>  1 file changed, 5 insertions(+), 5 deletions(-)

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>


Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [PATCH v2 1/2] migration/tls: fix inverted semantics in multifd_channel_connect

[Zheng Chuan]

Reviewed-by: Chuan Zheng <zhengchuan@huawei.com>

On 2021/2/9 18:42, Hao Wang wrote:
> Function multifd_channel_connect() return "true" to indicate failure,
> which is rather confusing. Fix that.
> 
> Signed-off-by: Hao Wang <wanghao232@huawei.com>
> ---
>  migration/multifd.c | 10 +++++-----
>  1 file changed, 5 insertions(+), 5 deletions(-)
> 
> diff --git a/migration/multifd.c b/migration/multifd.c
> index 1a1e589064..2a1ea85ade 100644
> --- a/migration/multifd.c
> +++ b/migration/multifd.c
> @@ -798,9 +798,9 @@ static bool multifd_channel_connect(MultiFDSendParams *p,
>                   * function after the TLS handshake,
>                   * so we mustn't call multifd_send_thread until then
>                   */
> -                return false;
> -            } else {
>                  return true;
> +            } else {
> +                return false;
>              }
>          } else {
>              /* update for tls qio channel */
> @@ -808,10 +808,10 @@ static bool multifd_channel_connect(MultiFDSendParams *p,
>              qemu_thread_create(&p->thread, p->name, multifd_send_thread, p,
>                                     QEMU_THREAD_JOINABLE);
>         }
> -       return false;
> +       return true;
>      }
>  
> -    return true;
> +    return false;
>  }
>  
>  static void multifd_new_send_channel_cleanup(MultiFDSendParams *p,
> @@ -844,7 +844,7 @@ static void multifd_new_send_channel_async(QIOTask *task, gpointer opaque)
>          p->c = QIO_CHANNEL(sioc);
>          qio_channel_set_delay(p->c, false);
>          p->running = true;
> -        if (multifd_channel_connect(p, sioc, local_err)) {
> +        if (!multifd_channel_connect(p, sioc, local_err)) {
>              goto cleanup;
>          }
>          return;
> 

-- 
Regards.
Chuan

[PATCH v2 2/2] migration/tls: add error handling in multifd_tls_handshake_thread

[Hao Wang]

If any error happens during multifd send thread creating (e.g. channel broke
because new domain is destroyed by the dst), multifd_tls_handshake_thread
may exit silently, leaving main migration thread hanging (ram_save_setup ->
multifd_send_sync_main -> qemu_sem_wait(&p->sem_sync)).
Fix that by adding error handling in multifd_tls_handshake_thread.

Signed-off-by: Hao Wang <wanghao232@huawei.com>
---
 migration/multifd.c | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)

diff --git a/migration/multifd.c b/migration/multifd.c
index 2a1ea85ade..03527c564c 100644
--- a/migration/multifd.c
+++ b/migration/multifd.c
@@ -739,7 +739,16 @@ static void multifd_tls_outgoing_handshake(QIOTask *task,
     } else {
         trace_multifd_tls_outgoing_handshake_complete(ioc);
     }
-    multifd_channel_connect(p, ioc, err);
+
+    if (!multifd_channel_connect(p, ioc, err)) {
+        /*
+         * Error happen, mark multifd_send_thread status as 'quit' although it
+         * is not created, and then tell who pay attention to me.
+         */
+        p->quit = true;
+        qemu_sem_post(&multifd_send_state->channels_ready);
+        qemu_sem_post(&p->sem_sync);
+    }
 }
 
 static void *multifd_tls_handshake_thread(void *opaque)
-- 
2.23.0

Re: [PATCH v2 2/2] migration/tls: add error handling in multifd_tls_handshake_thread

[Daniel P. Berrangé]

On Tue, Feb 09, 2021 at 06:42:37PM +0800, Hao Wang wrote:
> If any error happens during multifd send thread creating (e.g. channel broke
> because new domain is destroyed by the dst), multifd_tls_handshake_thread
> may exit silently, leaving main migration thread hanging (ram_save_setup ->
> multifd_send_sync_main -> qemu_sem_wait(&p->sem_sync)).
> Fix that by adding error handling in multifd_tls_handshake_thread.
> 
> Signed-off-by: Hao Wang <wanghao232@huawei.com>
> ---
>  migration/multifd.c | 11 ++++++++++-
>  1 file changed, 10 insertions(+), 1 deletion(-)

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>


Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [PATCH v2 2/2] migration/tls: add error handling in multifd_tls_handshake_thread

[Zheng Chuan]

Reviewed-by: Chuan Zheng <zhengchuan@huawei.com>

On 2021/2/9 18:42, Hao Wang wrote:
> If any error happens during multifd send thread creating (e.g. channel broke
> because new domain is destroyed by the dst), multifd_tls_handshake_thread
> may exit silently, leaving main migration thread hanging (ram_save_setup ->
> multifd_send_sync_main -> qemu_sem_wait(&p->sem_sync)).
> Fix that by adding error handling in multifd_tls_handshake_thread.
> 
> Signed-off-by: Hao Wang <wanghao232@huawei.com>
> ---
>  migration/multifd.c | 11 ++++++++++-
>  1 file changed, 10 insertions(+), 1 deletion(-)
> 
> diff --git a/migration/multifd.c b/migration/multifd.c
> index 2a1ea85ade..03527c564c 100644
> --- a/migration/multifd.c
> +++ b/migration/multifd.c
> @@ -739,7 +739,16 @@ static void multifd_tls_outgoing_handshake(QIOTask *task,
>      } else {
>          trace_multifd_tls_outgoing_handshake_complete(ioc);
>      }
> -    multifd_channel_connect(p, ioc, err);
> +
> +    if (!multifd_channel_connect(p, ioc, err)) {
> +        /*
> +         * Error happen, mark multifd_send_thread status as 'quit' although it
> +         * is not created, and then tell who pay attention to me.
> +         */
> +        p->quit = true;
> +        qemu_sem_post(&multifd_send_state->channels_ready);
> +        qemu_sem_post(&p->sem_sync);
> +    }
>  }
>  
>  static void *multifd_tls_handshake_thread(void *opaque)
> 

-- 
Regards.
Chuan

Re: [PATCH v2 0/2] add error handling in multifd_tls_handshake_thread

[Dr. David Alan Gilbert]

* Hao Wang (wanghao232@huawei.com) wrote:
> If any error happens during multifd send thread creating (e.g. channel broke
> because new domain is destroyed by the dst), multifd_tls_handshake_thread
> may exit silently, leaving main migration thread hanging (ram_save_setup ->
> multifd_send_sync_main -> qemu_sem_wait(&p->sem_sync)).
> Fix that by adding error handling in multifd_tls_handshake_thread.
> 
> Hao Wang (2):
>   migration/tls: fix inverted semantics in multifd_channel_connect
>   migration/tls: add error handling in multifd_tls_handshake_thread
> 
>  migration/multifd.c | 21 +++++++++++++++------
>  1 file changed, 15 insertions(+), 6 deletions(-)

Queued

> 
> -- 
> 2.23.0
> 
> 
-- 
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK