[PATCH for-8.0] aio-posix: fix race between epoll upgrade and aio_set_fd_handler()

[PATCH for-8.0] aio-posix: fix race between epoll upgrade and aio_set_fd_handler()

[Stefan Hajnoczi]

If another thread calls aio_set_fd_handler() while the IOThread event
loop is upgrading from ppoll(2) to epoll(7) then we might miss new
AioHandlers. The epollfd will not monitor the new AioHandler's fd,
resulting in hangs.

Take the AioHandler list lock while upgrading to epoll. This prevents
AioHandlers from changing while epoll is being set up. If we cannot lock
because we're in a nested event loop, then don't upgrade to epoll (it
will happen next time we're not in a nested call).

The downside to taking the lock is that the aio_set_fd_handler() thread
has to wait until the epoll upgrade is finished, which involves many
epoll_ctl(2) system calls. However, this scenario is rare and I couldn't
think of another solution that is still simple.

Reported-by: Qing Wang <qinwang@redhat.com>
Buglink: https://bugzilla.redhat.com/show_bug.cgi?id=2090998
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Fam Zheng <fam@euphon.net>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 util/fdmon-epoll.c | 25 ++++++++++++++++++-------
 1 file changed, 18 insertions(+), 7 deletions(-)

diff --git a/util/fdmon-epoll.c b/util/fdmon-epoll.c
index e11a8a022e..8f1323ab2c 100644
--- a/util/fdmon-epoll.c
+++ b/util/fdmon-epoll.c
@@ -127,6 +127,8 @@ static bool fdmon_epoll_try_enable(AioContext *ctx)
 
 bool fdmon_epoll_try_upgrade(AioContext *ctx, unsigned npfd)
 {
+    bool ok;
+
     if (ctx->epollfd < 0) {
         return false;
     }
@@ -136,14 +138,23 @@ bool fdmon_epoll_try_upgrade(AioContext *ctx, unsigned npfd)
         return false;
     }
 
-    if (npfd >= EPOLL_ENABLE_THRESHOLD) {
-        if (fdmon_epoll_try_enable(ctx)) {
-            return true;
-        } else {
-            fdmon_epoll_disable(ctx);
-        }
+    if (npfd < EPOLL_ENABLE_THRESHOLD) {
+        return false;
     }
-    return false;
+
+    /* The list must not change while we add fds to epoll */
+    if (!qemu_lockcnt_dec_if_lock(&ctx->list_lock)) {
+        return false;
+    }
+
+    ok = fdmon_epoll_try_enable(ctx);
+
+    qemu_lockcnt_unlock(&ctx->list_lock);
+
+    if (!ok) {
+        fdmon_epoll_disable(ctx);
+    }
+    return ok;
 }
 
 void fdmon_epoll_setup(AioContext *ctx)
-- 
2.39.2

Re: [PATCH for-8.0] aio-posix: fix race between epoll upgrade and aio_set_fd_handler()

[Paolo Bonzini]

[-- Attachment #1: Type: text/plain, Size: 565 bytes --]

Il mer 22 mar 2023, 15:55 Stefan Hajnoczi <stefanha@redhat.com> ha scritto:

> +    /* The list must not change while we add fds to epoll */
> +    if (!qemu_lockcnt_dec_if_lock(&ctx->list_lock)) {
> +        return false;
> +    }
> +
> +    ok = fdmon_epoll_try_enable(ctx);
> +
> +    qemu_lockcnt_unlock(&ctx->list_lock);
>

Shouldn't this be inc_and_unlock to balance the change made by dec_if_lock?

Paolo

+
> +    if (!ok) {
> +        fdmon_epoll_disable(ctx);
> +    }
> +    return ok;
>  }
>
>  void fdmon_epoll_setup(AioContext *ctx)
> --
> 2.39.2
>
>

[-- Attachment #2: Type: text/html, Size: 1228 bytes --]

Re: [PATCH for-8.0] aio-posix: fix race between epoll upgrade and aio_set_fd_handler()

[Stefan Hajnoczi]

[-- Attachment #1: Type: text/plain, Size: 607 bytes --]

On Thu, Mar 23, 2023 at 06:02:36AM +0100, Paolo Bonzini wrote:
> Il mer 22 mar 2023, 15:55 Stefan Hajnoczi <stefanha@redhat.com> ha scritto:
> 
> > +    /* The list must not change while we add fds to epoll */
> > +    if (!qemu_lockcnt_dec_if_lock(&ctx->list_lock)) {
> > +        return false;
> > +    }
> > +
> > +    ok = fdmon_epoll_try_enable(ctx);
> > +
> > +    qemu_lockcnt_unlock(&ctx->list_lock);
> >
> 
> Shouldn't this be inc_and_unlock to balance the change made by dec_if_lock?

Yes, the caller expects list_lock to still be incremented. Thanks for
catching this!

Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]