[PATCH] aio: Wake all waiters when destroying ctx

[PATCH] aio: Wake all waiters when destroying ctx

[Roland Dreier]

From: Roland Dreier <roland@purestorage.com>

The test program below will hang because io_getevents() uses
add_wait_queue_exclusive(), which means the wake_up() in io_destroy()
only wakes up one of the threads.  Fix this by using wake_up_all() in
the aio code paths where we want to make sure no one gets stuck.

	// t.c -- compile with gcc -lpthread -laio t.c

	#include <libaio.h>
	#include <pthread.h>
	#include <stdio.h>
	#include <unistd.h>

	static const int nthr = 2;

	void *getev(void *ctx)
	{
		struct io_event ev;
		io_getevents(ctx, 1, 1, &ev, NULL);
		printf("io_getevents returned\n");
		return NULL;
	}

	int main(int argc, char *argv[])
	{
		io_context_t ctx = 0;
		pthread_t thread[nthr];
		int i;

		io_setup(1024, &ctx);

		for (i = 0; i < nthr; ++i)
			pthread_create(&thread[i], NULL, getev, ctx);

		sleep(1);

		io_destroy(ctx);

		for (i = 0; i < nthr; ++i)
			pthread_join(thread[i], NULL);

		return 0;
	}

Cc: <stable@kernel.org>
Signed-off-by: Roland Dreier <roland@purestorage.com>
---
 fs/aio.c |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/fs/aio.c b/fs/aio.c
index 26869cd..88f0ed5 100644
--- a/fs/aio.c
+++ b/fs/aio.c
@@ -520,7 +520,7 @@ static inline void really_put_req(struct kioctx *ctx, struct kiocb *req)
 	ctx->reqs_active--;
 
 	if (unlikely(!ctx->reqs_active && ctx->dead))
-		wake_up(&ctx->wait);
+		wake_up_all(&ctx->wait);
 }
 
 static void aio_fput_routine(struct work_struct *data)
@@ -1229,7 +1229,7 @@ static void io_destroy(struct kioctx *ioctx)
 	 * by other CPUs at this point.  Right now, we rely on the
 	 * locking done by the above calls to ensure this consistency.
 	 */
-	wake_up(&ioctx->wait);
+	wake_up_all(&ioctx->wait);
 	put_ioctx(ioctx);	/* once for the lookup */
 }
 

Re: [PATCH] aio: Wake all waiters when destroying ctx

[Jeff Moyer]

Roland Dreier <roland@kernel.org> writes:

> From: Roland Dreier <roland@purestorage.com>
>
> The test program below will hang because io_getevents() uses
> add_wait_queue_exclusive(), which means the wake_up() in io_destroy()
> only wakes up one of the threads.  Fix this by using wake_up_all() in
> the aio code paths where we want to make sure no one gets stuck.

[snip]

> Cc: <stable@kernel.org>
> Signed-off-by: Roland Dreier <roland@purestorage.com>
> ---
>  fs/aio.c |    4 ++--
>  1 files changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/fs/aio.c b/fs/aio.c
> index 26869cd..88f0ed5 100644
> --- a/fs/aio.c
> +++ b/fs/aio.c
> @@ -520,7 +520,7 @@ static inline void really_put_req(struct kioctx *ctx, struct kiocb *req)
>  	ctx->reqs_active--;
>  
>  	if (unlikely(!ctx->reqs_active && ctx->dead))
> -		wake_up(&ctx->wait);
> +		wake_up_all(&ctx->wait);
>  }
>  
>  static void aio_fput_routine(struct work_struct *data)
> @@ -1229,7 +1229,7 @@ static void io_destroy(struct kioctx *ioctx)
>  	 * by other CPUs at this point.  Right now, we rely on the
>  	 * locking done by the above calls to ensure this consistency.
>  	 */
> -	wake_up(&ioctx->wait);
> +	wake_up_all(&ioctx->wait);
>  	put_ioctx(ioctx);	/* once for the lookup */
>  }
>  

Thanks, Roland, this looks good.

Reviewed-by: Jeff Moyer <jmoyer@redhat.com>

Re: [PATCH] aio: Wake all waiters when destroying ctx

[Andrew Morton]

On Fri, 11 Mar 2011 09:21:46 -0500
Jeff Moyer <jmoyer@redhat.com> wrote:

> Roland Dreier <roland@kernel.org> writes:
> 
> > From: Roland Dreier <roland@purestorage.com>
> >
> > The test program below will hang because io_getevents() uses
> > add_wait_queue_exclusive(), which means the wake_up() in io_destroy()
> > only wakes up one of the threads.  Fix this by using wake_up_all() in
> > the aio code paths where we want to make sure no one gets stuck.
> 
> [snip]
> 
> > Cc: <stable@kernel.org>
> > Signed-off-by: Roland Dreier <roland@purestorage.com>
> > ---
> >  fs/aio.c |    4 ++--
> >  1 files changed, 2 insertions(+), 2 deletions(-)
> >
> > diff --git a/fs/aio.c b/fs/aio.c
> > index 26869cd..88f0ed5 100644
> > --- a/fs/aio.c
> > +++ b/fs/aio.c
> > @@ -520,7 +520,7 @@ static inline void really_put_req(struct kioctx *ctx, struct kiocb *req)
> >  	ctx->reqs_active--;
> >  
> >  	if (unlikely(!ctx->reqs_active && ctx->dead))
> > -		wake_up(&ctx->wait);
> > +		wake_up_all(&ctx->wait);
> >  }
> >  
> >  static void aio_fput_routine(struct work_struct *data)
> > @@ -1229,7 +1229,7 @@ static void io_destroy(struct kioctx *ioctx)
> >  	 * by other CPUs at this point.  Right now, we rely on the
> >  	 * locking done by the above calls to ensure this consistency.
> >  	 */
> > -	wake_up(&ioctx->wait);
> > +	wake_up_all(&ioctx->wait);
> >  	put_ioctx(ioctx);	/* once for the lookup */
> >  }
> >  
> 
> Thanks, Roland, this looks good.
> 
> Reviewed-by: Jeff Moyer <jmoyer@redhat.com>

It's a fairly old bug (yes?) and we're presumably close to 2.6.38.  So
I scheduled the fix for 2.6.39 marked for backporting into 2.6.38.x and
earlier.

That may have been a bit over-cautious - feel free to argue ;)

Re: [PATCH] aio: Wake all waiters when destroying ctx

[Roland Dreier]

On Fri, Mar 11, 2011 at 4:19 PM, Andrew Morton
<akpm@linux-foundation.org> wrote:
> It's a fairly old bug (yes?) and we're presumably close to 2.6.38.  So
> I scheduled the fix for 2.6.39 marked for backporting into 2.6.38.x and
> earlier.

Yes, AFAICT it's been there approximately forever.

I agree that this is 2.6.39 stuff, and I included the stable@ cc so it gets
into 2.6.38.1.  Seems like the best course.

 - R.