Re: Patch for Retry based AIO-DIO (Was AIO and DIO testing on 2.6.0-test7-mm1)

[Suparna Bhattacharya <suparna@in.ibm.com>]

On Wed, Oct 22, 2003 at 05:40:32PM -0700, Daniel McNeil wrote:
> Suparna and Andrew,
> 
> I've been doing more testing using the test programs I wrote to 
> try and hit the AIO verses buffered read race conditions.
> 
> I tested 2.6.0-test8, 2.6.0-test8-mm1+(your first incomplete fix) and
> 2.6.0-test8-mm1+aio-dio-retry patch.  I used my test programs
> (http://developer.osdl.org/daniel/AIO/TESTS/) by doing:
> 
> Run "dirty" program which allocates and writes 0xaa to a file and then
> 	frees the space.
> Run "dio_sparse" or "aiodio-sparse - which creates "file", truncates it
> 	up to 64MB and then writes zeros into the holes (using DIO or
> 	AIO+DIO).  At same time, a forked child is reading the file
> 	looking for non-zero data.
> rm "file"
> 
> On 2.6.0-test8
> ==============
> I hit the race condition and see uninitialized data:
> ~/AIO/TESTS/dio_sparse
> non zero buffer at buf[4] => 0xaaaaaaaa,aaaaaaaa,aaaaaaaa,aaaaaaaa
> non-zero read at offset 24182785
> 
>  ~/AIO/TESTS/aiodio_sparse
> non zero buffer at buf[4] -> 0xaaaaaaaa,aaaaaaaa,aaaaaaaa,aaaaaaaa
> non-zero read at offset 8323062
> 
> 
> On 2.6.0-test8-mm1+1st-direct-io-aio_complete patch and
>    2.6.0-test8-mm1+aio-dio-retry patch
> 
> I never see uninitialized data.

That's good news.

You seem to be able to run test8-mm1 just fine; I have been
running into strange oops on syscall return for io_getevents :(
- haven't seen this before.
What library and header files are you using for libaio ? Do you have
4G-4G turned on in your build ?

> 
> Reading over your description and looking over the code, I'm pretty
> sure I see the races you are talking about but had some questions:
> 
> 1) DIO file extends.  So are you saying that in direct_io_worker()
>    we were dropping i_sem and then waiting for the i/o to complete,
>    but since i_sem was dropped there could be another write exposing
>    the uninitialized data still in process of being written?
>    
>    If yes, then wouldn't the intermediate write be responsible for
>    zeroing all data between the last isize and the new isize?

I think it would have to initialize all the data involved in the 
write but could just leave a hole from the last isize to the 
start of this write. It doesn't expect uninstantiated blocks to 
exist in that hole, so shouldn't have to worry about zeroing 
them.

> 
> 2) aio-dio extends
> 
>    Are you saying that in __generic_file_aio_write_nolock()
>    that generic_file_direct_IO() returns the number of bytes
>    that are being AIO written but not complete and we were updating
>    i_size write after that?  (just trying to understand the code).

Yes. And that leaving uninitialised data exposed.

Regards
Suparna

-
Suparna Bhattacharya (suparna@in.ibm.com)
Linux Technology Center
IBM Software Labs, India