[PATCH vOther2 1/1] qemu-nbd: Restore "qemu-nbd -v --fork" output

[PATCH vOther2 1/1] qemu-nbd: Restore "qemu-nbd -v --fork" output

[Denis V. Lunev]

Closing stderr earlier is good for daemonized qemu-nbd under ssh
earlier, but breaks the case where -v is being used to track what is
happening in the server, as in iotest 233.

When we know we are verbose, we should preserve original stderr and
restore it once the setup stage is done. This commit restores the
original behavior with -v option. In this case original output
inside the test is kept intact.

Reported-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Denis V. Lunev <den@openvz.org>
CC: Eric Blake <eblake@redhat.com>
CC: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
CC: Hanna Reitz <hreitz@redhat.com>
CC: Mike Maslenkin <mike.maslenkin@gmail.com>
Fixes: 5c56dd27a2 ("qemu-nbd: fix regression with qemu-nbd --fork run over ssh")
---
Changes from v1:
* fixed compilation with undefined HAVE_NBD_DEVICE, thanks to Mike Maslenkin

 qemu-nbd.c | 39 ++++++++++++++++++++++++++++-----------
 1 file changed, 28 insertions(+), 11 deletions(-)

diff --git a/qemu-nbd.c b/qemu-nbd.c
index aaccaa3318..19a4147d24 100644
--- a/qemu-nbd.c
+++ b/qemu-nbd.c
@@ -253,6 +253,13 @@ static int qemu_nbd_client_list(SocketAddress *saddr, QCryptoTLSCreds *tls,
 }
 
 
+struct NbdClientOpts {
+    char *device;
+    bool fork_process;
+    bool verbose;
+    int stderr;
+};
+
 #if HAVE_NBD_DEVICE
 static void *show_parts(void *arg)
 {
@@ -271,12 +278,6 @@ static void *show_parts(void *arg)
     return NULL;
 }
 
-struct NbdClientOpts {
-    char *device;
-    bool fork_process;
-    bool verbose;
-};
-
 static void *nbd_client_thread(void *arg)
 {
     struct NbdClientOpts *opts = arg;
@@ -323,11 +324,14 @@ static void *nbd_client_thread(void *arg)
                 opts->device, srcpath);
     } else {
         /* Close stderr so that the qemu-nbd process exits.  */
-        if (dup2(STDOUT_FILENO, STDERR_FILENO) < 0) {
+        if (dup2(opts->stderr, STDERR_FILENO) < 0) {
             error_report("Could not set stderr to /dev/null: %s",
                          strerror(errno));
             exit(EXIT_FAILURE);
         }
+        if (opts->stderr != STDOUT_FILENO) {
+            close(opts->stderr);
+        }
     }
 
     if (nbd_client(fd) < 0) {
@@ -589,9 +593,9 @@ int main(int argc, char **argv)
     const char *pid_file_name = NULL;
     const char *selinux_label = NULL;
     BlockExportOptions *export_opts;
-#if HAVE_NBD_DEVICE
-    struct NbdClientOpts opts;
-#endif
+    struct NbdClientOpts opts = {
+        .stderr = STDOUT_FILENO,
+    };
 
 #ifdef CONFIG_POSIX
     os_setup_early_signal_handling();
@@ -944,6 +948,15 @@ int main(int argc, char **argv)
 
             close(stderr_fd[0]);
 
+            /* Remember parent's stderr if we will be restoring it. */
+            if (verbose /* fork_process is set */) {
+                opts.stderr = dup(STDERR_FILENO);
+                if (opts.stderr < 0) {
+                    error_report("Could not dup stdedd: %s", strerror(errno));
+                    exit(EXIT_FAILURE);
+                }
+            }
+
             ret = qemu_daemon(1, 0);
             saved_errno = errno;    /* dup2 will overwrite error below */
 
@@ -1152,6 +1165,7 @@ int main(int argc, char **argv)
             .device = device,
             .fork_process = fork_process,
             .verbose = verbose,
+            .stderr = STDOUT_FILENO,
         };
 
         ret = pthread_create(&client_thread, NULL, nbd_client_thread, &opts);
@@ -1180,11 +1194,14 @@ int main(int argc, char **argv)
     }
 
     if (fork_process) {
-        if (dup2(STDOUT_FILENO, STDERR_FILENO) < 0) {
+        if (dup2(opts.stderr, STDERR_FILENO) < 0) {
             error_report("Could not set stderr to /dev/null: %s",
                          strerror(errno));
             exit(EXIT_FAILURE);
         }
+        if (opts.stderr != STDOUT_FILENO) {
+            close(opts.stderr);
+        }
     }
 
     state = RUNNING;
-- 
2.34.1

Re: [PATCH vOther2 1/1] qemu-nbd: Restore "qemu-nbd -v --fork" output

[Eric Blake]

On Fri, Aug 25, 2023 at 10:08:38PM +0200, Denis V. Lunev wrote:
> Closing stderr earlier is good for daemonized qemu-nbd under ssh
> earlier, but breaks the case where -v is being used to track what is
> happening in the server, as in iotest 233.

Keeping the iotest output unchanged is a nice effect, even if it
requires a bit more code, so I'm leaning towards taking your patch.

> 
> When we know we are verbose, we should preserve original stderr and
> restore it once the setup stage is done. This commit restores the
> original behavior with -v option. In this case original output
> inside the test is kept intact.
> 
> Reported-by: Kevin Wolf <kwolf@redhat.com>
> Signed-off-by: Denis V. Lunev <den@openvz.org>
> CC: Eric Blake <eblake@redhat.com>
> CC: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
> CC: Hanna Reitz <hreitz@redhat.com>
> CC: Mike Maslenkin <mike.maslenkin@gmail.com>
> Fixes: 5c56dd27a2 ("qemu-nbd: fix regression with qemu-nbd --fork run over ssh")
> ---
> Changes from v1:
> * fixed compilation with undefined HAVE_NBD_DEVICE, thanks to Mike Maslenkin

> @@ -323,11 +324,14 @@ static void *nbd_client_thread(void *arg)
>                  opts->device, srcpath);
>      } else {
>          /* Close stderr so that the qemu-nbd process exits.  */
> -        if (dup2(STDOUT_FILENO, STDERR_FILENO) < 0) {
> +        if (dup2(opts->stderr, STDERR_FILENO) < 0) {
>              error_report("Could not set stderr to /dev/null: %s",

Both my patch and yours have a slight inaccuracy here: when -v is in
use, failure to dup2() is not a failure to set stderr to /dev/null.
Maybe we can reword it as "Could not release pipe to parent: %s", as
that is the other intentional side effect of the dup2()?

>                           strerror(errno));
>              exit(EXIT_FAILURE);
>          }
> +        if (opts->stderr != STDOUT_FILENO) {
> +            close(opts->stderr);

As long as we are checking dup2() for (unlikely) failure, we should
probably be doing the same for close().

> +        }
>      }
>  
>      if (nbd_client(fd) < 0) {
> @@ -589,9 +593,9 @@ int main(int argc, char **argv)
>      const char *pid_file_name = NULL;
>      const char *selinux_label = NULL;
>      BlockExportOptions *export_opts;
> -#if HAVE_NBD_DEVICE
> -    struct NbdClientOpts opts;
> -#endif
> +    struct NbdClientOpts opts = {
> +        .stderr = STDOUT_FILENO,
> +    };
>  
>  #ifdef CONFIG_POSIX
>      os_setup_early_signal_handling();
> @@ -944,6 +948,15 @@ int main(int argc, char **argv)
>  
>              close(stderr_fd[0]);
>  
> +            /* Remember parent's stderr if we will be restoring it. */
> +            if (verbose /* fork_process is set */) {
> +                opts.stderr = dup(STDERR_FILENO);
> +                if (opts.stderr < 0) {
> +                    error_report("Could not dup stdedd: %s", strerror(errno));

s/stdedd/stderr/

> +                    exit(EXIT_FAILURE);
> +                }
> +            }
> +
>              ret = qemu_daemon(1, 0);
>              saved_errno = errno;    /* dup2 will overwrite error below */
>  
> @@ -1152,6 +1165,7 @@ int main(int argc, char **argv)
>              .device = device,
>              .fork_process = fork_process,
>              .verbose = verbose,
> +            .stderr = STDOUT_FILENO,

Huh. This looks redundant to pre-initializing .stderr above; but since
it is using a struct assignment, we do have to provide it again to
avoid the compiler setting unmentioned fields to zero-initialization.

If we are going to unconditionally have opts in scope, even when not
passing it to pthread_create(), maybe we could just directly assign to
opts.device and drop the local variable device (and so forth), instead
of first storing into device only to later copy it to opts.device.
But that would make this patch bigger, so I'm not sure it is worth it.

>          };
>  
>          ret = pthread_create(&client_thread, NULL, nbd_client_thread, &opts);
> @@ -1180,11 +1194,14 @@ int main(int argc, char **argv)
>      }
>  
>      if (fork_process) {
> -        if (dup2(STDOUT_FILENO, STDERR_FILENO) < 0) {
> +        if (dup2(opts.stderr, STDERR_FILENO) < 0) {
>              error_report("Could not set stderr to /dev/null: %s",
>                           strerror(errno));

Another spot where the error message is not entirely accurate,

>              exit(EXIT_FAILURE);
>          }
> +        if (opts.stderr != STDOUT_FILENO) {
> +            close(opts.stderr);

and another spot where we should be checking for close() failure.

> +        }
>      }
>  
>      state = RUNNING;
> -- 
> 2.34.1
> 

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.
Virtualization:  qemu.org | libguestfs.org

Re: [PATCH vOther2 1/1] qemu-nbd: Restore "qemu-nbd -v --fork" output

[Michael Tokarev]

25.08.2023 23:08, Denis V. Lunev wrote:
> +            /* Remember parent's stderr if we will be restoring it. */
> +            if (verbose /* fork_process is set */) {
> +                opts.stderr = dup(STDERR_FILENO);
> +                if (opts.stderr < 0) {
> +                    error_report("Could not dup stdedd: %s", strerror(errno));
> +                    exit(EXIT_FAILURE);
> +                }
> +            }
> +
>               ret = qemu_daemon(1, 0);

I haven't looked closely to this development.

To me it all looks.. backwards.

Instead of saving stderr around qemu_daemon() call, it might be more
productive to tell qemu_daemon() to stop redirecting stderr (and maybe
instrumenting it to do so).

Besides, qemu has 2 daemon implementations, one is qemu_daemon()
in util/oslib-posix.c and another is os_daemonize() in os-posix.c.
Note os_daemonize() does the right thing wrt logging already, -

         /* In case -D is given do not redirect stderr to /dev/null */
         if (!qemu_log_enabled()) {
             dup2(fd, 2);
         }

but I guess nbd does not use qemu_log_enabled() et al.

Also, qemu-nbd can benefit from using -runas/-chroot too.

Ideally this whole thing should be consolidated.  I already took a
step towards this by moving softmmu-specific stuff from os-posix.c
to softmmu/, this work should continue. When it's done, we can
revert this band-aid change for a real solution.

Thanks,

/mjt

Re: [PATCH vOther2 1/1] qemu-nbd: Restore "qemu-nbd -v --fork" output

[Eric Blake]

On Sun, Sep 10, 2023 at 11:40:25AM +0300, Michael Tokarev wrote:
> 25.08.2023 23:08, Denis V. Lunev wrote:
> > +            /* Remember parent's stderr if we will be restoring it. */
> > +            if (verbose /* fork_process is set */) {
> > +                opts.stderr = dup(STDERR_FILENO);
> > +                if (opts.stderr < 0) {
> > +                    error_report("Could not dup stdedd: %s", strerror(errno));
> > +                    exit(EXIT_FAILURE);
> > +                }
> > +            }
> > +
> >               ret = qemu_daemon(1, 0);
> 
> I haven't looked closely to this development.
> 
> To me it all looks.. backwards.
> 
> Instead of saving stderr around qemu_daemon() call, it might be more
> productive to tell qemu_daemon() to stop redirecting stderr (and maybe
> instrumenting it to do so).

I tried to do that in an earlier revision of this patch, but it
changed iotest output.  We've already had a lot of churn on patches
that were supposed to fix a regression but in turn caused another
regression.

> 
> Besides, qemu has 2 daemon implementations, one is qemu_daemon()
> in util/oslib-posix.c and another is os_daemonize() in os-posix.c.
> Note os_daemonize() does the right thing wrt logging already, -
> 
>         /* In case -D is given do not redirect stderr to /dev/null */
>         if (!qemu_log_enabled()) {
>             dup2(fd, 2);
>         }
> 
> but I guess nbd does not use qemu_log_enabled() et al.
> 
> Also, qemu-nbd can benefit from using -runas/-chroot too.
> 
> Ideally this whole thing should be consolidated.  I already took a
> step towards this by moving softmmu-specific stuff from os-posix.c
> to softmmu/, this work should continue. When it's done, we can
> revert this band-aid change for a real solution.

Indeed, there may still be further cleanups to do once the os-posix.c
cleanups are in.

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.
Virtualization:  qemu.org | libguestfs.org