[Qemu-devel] [PATCH] nbd: do not hang nbd_wr_syncv if outside a coroutine and no available data

[Qemu-devel] [PATCH] nbd: do not hang nbd_wr_syncv if outside a coroutine and no available data

[Paolo Bonzini]

Until commit 1c778ef7 ("nbd: convert to using I/O channels for actual
socket I/O", 2016-02-16), nbd_wr_sync returned -EAGAIN this scenario.
nbd_reply_ready required these semantics because it has two conflicting
requirements:

1) if a reply can be received on the socket, nbd_reply_ready needs
to read the header outside coroutine context to identify _which_
coroutine to enter to process the rest of the reply

2) on the other hand, nbd_reply_ready can find a false positive if
another thread (e.g. a VCPU thread running aio_poll) sneaks in and
calls nbd_reply_ready too.  In this case nbd_reply_ready does nothing
and expects nbd_wr_syncv to return -EAGAIN.

Currently, the solution to the first requirement is to wait in the very
rare case of a read() that doesn't retrieve the reply header in its
entirety; this is what nbd_wr_syncv does by calling qio_channel_wait().
However, the unconditional call to qio_channel_wait() breaks the second
requirement.  To fix this, the patch makes nbd_wr_syncv return -EAGAIN
if done is zero, similar to the code before commit 1c778ef7.

This is okay because NBD client-side negotiation is the only other case
that calls nbd_wr_syncv outside a coroutine, and it places the socket
in blocking mode.  On the other hand, it is a bit unpleasant to put
this in nbd_wr_syncv(), because the function is used by both client
and server.

The full fix would be to add a counter to NbdClientSession for how
many bytes have been filled in s->reply.  Then a reply can be filled
by multiple separate invocations of nbd_reply_ready and the
qio_channel_wait() call can be removed completely.  Something to
consider for 2.7...

Reported-by: Changlong Xie <xiecl.fnst@cn.fujitsu.com>
Cc: Daniel P. Berrange <berrange@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 nbd/common.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/nbd/common.c b/nbd/common.c
index a44718c..8ddb2dd 100644
--- a/nbd/common.c
+++ b/nbd/common.c
@@ -50,9 +50,12 @@ ssize_t nbd_wr_syncv(QIOChannel *ioc,
                  * qio_channel_yield() that works with AIO contexts
                  * and consider using that in this branch */
                 qemu_coroutine_yield();
-            } else {
+            } else if (done) {
+                /* XXX this is needed by nbd_reply_ready.  */
                 qio_channel_wait(ioc,
                                  do_read ? G_IO_IN : G_IO_OUT);
+            } else {
+                return -EAGAIN;
             }
             continue;
         }
-- 
2.5.5

Re: [Qemu-devel] [PATCH] nbd: do not hang nbd_wr_syncv if outside a coroutine and no available data

[Daniel P. Berrange]

On Thu, Apr 07, 2016 at 01:44:55PM +0200, Paolo Bonzini wrote:
> Until commit 1c778ef7 ("nbd: convert to using I/O channels for actual
> socket I/O", 2016-02-16), nbd_wr_sync returned -EAGAIN this scenario.
> nbd_reply_ready required these semantics because it has two conflicting
> requirements:
> 
> 1) if a reply can be received on the socket, nbd_reply_ready needs
> to read the header outside coroutine context to identify _which_
> coroutine to enter to process the rest of the reply
> 
> 2) on the other hand, nbd_reply_ready can find a false positive if
> another thread (e.g. a VCPU thread running aio_poll) sneaks in and
> calls nbd_reply_ready too.  In this case nbd_reply_ready does nothing
> and expects nbd_wr_syncv to return -EAGAIN.
> 
> Currently, the solution to the first requirement is to wait in the very
> rare case of a read() that doesn't retrieve the reply header in its
> entirety; this is what nbd_wr_syncv does by calling qio_channel_wait().
> However, the unconditional call to qio_channel_wait() breaks the second
> requirement.  To fix this, the patch makes nbd_wr_syncv return -EAGAIN
> if done is zero, similar to the code before commit 1c778ef7.
> 
> This is okay because NBD client-side negotiation is the only other case
> that calls nbd_wr_syncv outside a coroutine, and it places the socket
> in blocking mode.  On the other hand, it is a bit unpleasant to put
> this in nbd_wr_syncv(), because the function is used by both client
> and server.
> 
> The full fix would be to add a counter to NbdClientSession for how
> many bytes have been filled in s->reply.  Then a reply can be filled
> by multiple separate invocations of nbd_reply_ready and the
> qio_channel_wait() call can be removed completely.  Something to
> consider for 2.7...
> 
> Reported-by: Changlong Xie <xiecl.fnst@cn.fujitsu.com>
> Cc: Daniel P. Berrange <berrange@redhat.com>
> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
> ---
>  nbd/common.c | 5 ++++-
>  1 file changed, 4 insertions(+), 1 deletion(-)

Reviewed-by: Daniel P. berrange <berrange@redhat.com>

Regards,
Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|

Re: [Qemu-devel] [PATCH] nbd: do not hang nbd_wr_syncv if outside a coroutine and no available data

[Changlong Xie]

On 04/07/2016 07:44 PM, Paolo Bonzini wrote:
> Until commit 1c778ef7 ("nbd: convert to using I/O channels for actual
> socket I/O", 2016-02-16), nbd_wr_sync returned -EAGAIN this scenario.
> nbd_reply_ready required these semantics because it has two conflicting
> requirements:
>
> 1) if a reply can be received on the socket, nbd_reply_ready needs
> to read the header outside coroutine context to identify _which_
> coroutine to enter to process the rest of the reply
>
> 2) on the other hand, nbd_reply_ready can find a false positive if
> another thread (e.g. a VCPU thread running aio_poll) sneaks in and
> calls nbd_reply_ready too.  In this case nbd_reply_ready does nothing
> and expects nbd_wr_syncv to return -EAGAIN.
>
> Currently, the solution to the first requirement is to wait in the very
> rare case of a read() that doesn't retrieve the reply header in its
> entirety; this is what nbd_wr_syncv does by calling qio_channel_wait().
> However, the unconditional call to qio_channel_wait() breaks the second
> requirement.  To fix this, the patch makes nbd_wr_syncv return -EAGAIN
> if done is zero, similar to the code before commit 1c778ef7.
>
> This is okay because NBD client-side negotiation is the only other case
> that calls nbd_wr_syncv outside a coroutine, and it places the socket
> in blocking mode.  On the other hand, it is a bit unpleasant to put
> this in nbd_wr_syncv(), because the function is used by both client
> and server.
>
> The full fix would be to add a counter to NbdClientSession for how
> many bytes have been filled in s->reply.  Then a reply can be filled
> by multiple separate invocations of nbd_reply_ready and the
> qio_channel_wait() call can be removed completely.  Something to
> consider for 2.7...
>
> Reported-by: Changlong Xie <xiecl.fnst@cn.fujitsu.com>
> Cc: Daniel P. Berrange <berrange@redhat.com>
> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
> ---
>   nbd/common.c | 5 ++++-
>   1 file changed, 4 insertions(+), 1 deletion(-)
>
> diff --git a/nbd/common.c b/nbd/common.c
> index a44718c..8ddb2dd 100644
> --- a/nbd/common.c
> +++ b/nbd/common.c
> @@ -50,9 +50,12 @@ ssize_t nbd_wr_syncv(QIOChannel *ioc,
>                    * qio_channel_yield() that works with AIO contexts
>                    * and consider using that in this branch */
>                   qemu_coroutine_yield();
> -            } else {
> +            } else if (done) {
> +                /* XXX this is needed by nbd_reply_ready.  */
>                   qio_channel_wait(ioc,
>                                    do_read ? G_IO_IN : G_IO_OUT);
> +            } else {
> +                return -EAGAIN;
>               }

With this patch, nbd works well now.

Thanks
	-Xie

>               continue;
>           }
>