Re: [Qemu-devel] [RFC 3/5] nbd: Use aio_set_fd_handler2()

[Stefan Hajnoczi <stefanha@redhat.com>]

On Wed, Jun 04, 2014 at 08:02:06PM +0200, Paolo Bonzini wrote:
> Il 04/06/2014 14:37, Stefan Hajnoczi ha scritto:
> >Why is this design cleaner?  Because NBD code doesn't have to worry
> >about fd handlers.  It uses straightforward coroutine send/recv for
> >socket I/O inside nbd_read_req() and nbd_write_resp().  It's easy to see
> >that only one coroutine receives from the socket and that only one
> >coroutine writes to the socket.
> 
> I don't understand how this would work without managing fd handlers.

fd handlers still need to be managed, but not by NBD code.  They must be
managed by coroutine recv/send utility functions.  In other words, fd
handlers are used locally, not globally.

def co_recv(fd, buf):
    while True:
        nbytes = recv(fd, buf, len(buf))
        if nbytes == -1:
            if errno == EINTR:
                continue
            if errno == EAGAIN or errno == EWOULDBLOCK:
                aio_set_fd_read_handler(fd, co_recv_cb)
                qemu_coroutine_yield()
                aio_set_fd_read_handler(fd, NULL)
                continue
        return nbytes

The send function is similar.

This does require an extension to the AioContext API.  We need to be
able to modify the read/write callback independently without clobbering
the other callback.  This way full duplex I/O is possible.

> - If you don't want to receive anything (because you reached the maximum
> queue depth), and the client sends something, the read handler will busy
> wait.  The current code solves it with can_read; you could do it by enabling
> or disabling the read handler as you need, but the idea is the same.
> 
> - If you're not sending anything, a socket that has a write handler will
> have POLLOUT continuously signaled and again you'd busy wait.  Since there
> is no can_write, nbd.c instead enables/disables the write handler around
> nbd_co_send_reply.

You only install an fd handler when you want to read/write.  This does
mean that the request coroutine needs to be woken up by the response
coroutine if we were at maximum queue depth.

Stefan