[Qemu-devel] [PATCH] io/channel-command: Do not kill the child process after closing the pipe

[Qemu-devel] [PATCH] io/channel-command: Do not kill the child process after closing the pipe

[Thomas Huth]

We are currently facing some migration failure on s390x when running
certain avocado-vt tests, e.g. when running the test
type_specific.io-github-autotest-qemu.migrate.with_reboot.exec.gzip_exec.
This test is using 'migrate -d "exec:nc localhost 5200"' for the migration.
The problem is detected at the receiving side, where the migration stream
apparently ends too early. However, the cause for the problem is at the
sending side: After writing the migration stream into the pipe to netcat,
the source QEMU calls qio_channel_command_close() which closes the pipe
and immediately (!) kills the child process afterwards (via the function
qio_channel_command_abort()). So if the  sending netcat did not read the
final bytes from the pipe yet, or  if it did not manage to send out all
its buffers yet, it is killed before the whole migration stream is passed
to the destination side.

QEMU can not know how much time is required by the child process to send
over all migration data, so we should not kill it, neither directly nor
after a delay. Let's simply wait for the child process to exit gracefully
instead (this was also the behaviour of pclose() that was used in "exec:"
migration before the QIOChannel rework).

Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 io/channel-command.c | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)

diff --git a/io/channel-command.c b/io/channel-command.c
index 319c5ed..3e7eb17 100644
--- a/io/channel-command.c
+++ b/io/channel-command.c
@@ -301,6 +301,9 @@ static int qio_channel_command_close(QIOChannel *ioc,
 {
     QIOChannelCommand *cioc = QIO_CHANNEL_COMMAND(ioc);
     int rv = 0;
+#ifndef WIN32
+    pid_t wp;
+#endif
 
     /* We close FDs before killing, because that
      * gives a better chance of clean shutdown
@@ -315,11 +318,18 @@ static int qio_channel_command_close(QIOChannel *ioc,
         rv = -1;
     }
     cioc->writefd = cioc->readfd = -1;
+
 #ifndef WIN32
-    if (qio_channel_command_abort(cioc, errp) < 0) {
+    do {
+        wp = waitpid(cioc->pid, NULL, 0);
+    } while (wp == (pid_t)-1 && errno == EINTR);
+    if (wp == (pid_t)-1) {
+        error_setg_errno(errp, errno, "Failed to wait for pid %llu",
+                         (unsigned long long)cioc->pid);
         return -1;
     }
 #endif
+
     if (rv < 0) {
         error_setg_errno(errp, errno, "%s",
                          "Unable to close command");
-- 
1.8.3.1

Re: [Qemu-devel] [PATCH] io/channel-command: Do not kill the child process after closing the pipe

[Daniel P. Berrangé]

On Wed, Feb 14, 2018 at 05:09:35PM +0100, Thomas Huth wrote:
> We are currently facing some migration failure on s390x when running
> certain avocado-vt tests, e.g. when running the test
> type_specific.io-github-autotest-qemu.migrate.with_reboot.exec.gzip_exec.
> This test is using 'migrate -d "exec:nc localhost 5200"' for the migration.
> The problem is detected at the receiving side, where the migration stream
> apparently ends too early. However, the cause for the problem is at the
> sending side: After writing the migration stream into the pipe to netcat,
> the source QEMU calls qio_channel_command_close() which closes the pipe
> and immediately (!) kills the child process afterwards (via the function
> qio_channel_command_abort()). So if the  sending netcat did not read the
> final bytes from the pipe yet, or  if it did not manage to send out all
> its buffers yet, it is killed before the whole migration stream is passed
> to the destination side.
> 
> QEMU can not know how much time is required by the child process to send
> over all migration data, so we should not kill it, neither directly nor
> after a delay. Let's simply wait for the child process to exit gracefully
> instead (this was also the behaviour of pclose() that was used in "exec:"
> migration before the QIOChannel rework).
> 
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
>  io/channel-command.c | 12 +++++++++++-
>  1 file changed, 11 insertions(+), 1 deletion(-)
> 
> diff --git a/io/channel-command.c b/io/channel-command.c
> index 319c5ed..3e7eb17 100644
> --- a/io/channel-command.c
> +++ b/io/channel-command.c
> @@ -301,6 +301,9 @@ static int qio_channel_command_close(QIOChannel *ioc,
>  {
>      QIOChannelCommand *cioc = QIO_CHANNEL_COMMAND(ioc);
>      int rv = 0;
> +#ifndef WIN32
> +    pid_t wp;
> +#endif
>  
>      /* We close FDs before killing, because that
>       * gives a better chance of clean shutdown
> @@ -315,11 +318,18 @@ static int qio_channel_command_close(QIOChannel *ioc,
>          rv = -1;
>      }
>      cioc->writefd = cioc->readfd = -1;
> +
>  #ifndef WIN32
> -    if (qio_channel_command_abort(cioc, errp) < 0) {
> +    do {
> +        wp = waitpid(cioc->pid, NULL, 0);
> +    } while (wp == (pid_t)-1 && errno == EINTR);
> +    if (wp == (pid_t)-1) {
> +        error_setg_errno(errp, errno, "Failed to wait for pid %llu",
> +                         (unsigned long long)cioc->pid);
>          return -1;
>      }
>  #endif
> +
>      if (rv < 0) {
>          error_setg_errno(errp, errno, "%s",
>                           "Unable to close command");

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>

I'll queue this as i had a PR pending...

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|