Re: [patch, v3] add an aio test which closes the fd before destroying the ioctx

[Dave Chinner <david@fromorbit.com>]

On Mon, Aug 25, 2014 at 12:50:43PM -0400, Benjamin LaHaise wrote:
> On Thu, Aug 21, 2014 at 09:57:50AM -0700, Zach Brown wrote:
> > On Wed, Aug 20, 2014 at 07:43:19PM -0400, Jeff Moyer wrote:
> > > Hi, Dave,
> > > 
> > > Dave Chinner <david@fromorbit.com> writes:
> > > 
> > > > IOWs, we now have two AIO+DIO tests showing the same symptoms that
> > > > no other tests show. This tends to point at AIO not being fully
> > > > cleaned up and completely freed by the time the processes
> > > > dispatching it have exit()d. This failure generally occurs when
> > > > there is other load on the system/disks backing the test VM (e.g.
> > > > running xfstests in multiple VMs at the same time) so I suspect it
> > > > has to do with IO completion taking a long time.
> > > 
> > > Process exit waits for all outstanding I/O, but maybe it's an rcu thing.
> > 
> > I thought it did too but it doesn't look like upstream exit_aio() is
> > waiting for iocbs to complete.
> > 
> > Ben, are you digging in to this?  Want me to throw something together?
> 
> Something like the following should fix it.  This is only lightly tested.  
> Does someone already have a simple test case we can add to the libaio test 
> suite to verify this behaviour?  I'm assuming that waiting for one ioctx 
> at a time is sufficient and we don't need to parallelise cancellation at 
> exit.

both xfstests::generic/208 and xfstests::generic/323 reproduce this.
I'm seeing a long term failure rate (i.e. over the past year) of
around 15% for generic/208 on my test VMs....

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com