Re: [PATCH 2.5.73-mm2] aio O_DIRECT no readahead

[Suparna Bhattacharya <suparna@in.ibm.com>]

On Mon, Jun 30, 2003 at 06:19:51PM -0700, Daniel McNeil wrote:
> More testing on AIO with O_DIRECT and /dev/raw/.  Doing AIO reads was
> using a lot more cpu time than using dd on the raw partition even with
> the io_queue_wait patch.  Found out that aio is doing readahead even
> for O_DIRECT.  Here's a patch that fixes it.  Here are some output

That was a good catch. Thanks ! 

> from time on reading a 90mb partition on a raw device with 32k i/o:
> 
> CMD             Kernel                  I/O     real    user    sys
> RUN             version                 size
> ===             ======                  ===     ====    ====    ===
> dd              2.5.73-mm2:             32k     2.514s  0.000s  0.200s
> aio (1 iocb)    2.5.73-mm2:             32k     3.408s  1.120s  2.250s
> aio (32 iocb)   2.5.73-mm2:             32k     2.994s  0.830s  2.160s
> aio (1 iocb)    2.5.73-mm2+patch.aiodio 32k     2.509s  0.850s  1.660s
> aio (1 iocb)    2.5.73-mm2+patch.aiodio
>                    +patch.io_queue_wait 32k     2.566s  0.010s  0.240s
> aio (32 iocb)   2.5.73-mm2+patch.aiodio
>                    +patch.io_queue_wait 32k     2.465s  0.010s  0.080s
> 
> With this patch and my previous io_queue_wait patch the cpu time
> drops down to very little when doing AIO with O_DIRECT.
> 
> Daniel McNeil <daniel@osdl.org>

> --- linux-2.5.73-mm2/fs/aio.c	2003-06-30 16:39:15.874216228 -0700
> +++ linux-2.5.73-mm2.aiodio/fs/aio.c	2003-06-30 15:38:46.000000000 -0700
> @@ -1380,15 +1380,20 @@ ssize_t aio_setup_iocb(struct kiocb *kio
>  			break;
>  		ret = -EINVAL;
>  		if (file->f_op->aio_read) {
> -			struct address_space *mapping =
> -				file->f_dentry->d_inode->i_mapping;
> -			unsigned long index = kiocb->ki_pos >> PAGE_CACHE_SHIFT;
> -			unsigned long end = (kiocb->ki_pos + kiocb->ki_left)
> -				>> PAGE_CACHE_SHIFT;
> -
> -			for (; index < end; index++) {
> -				page_cache_readahead(mapping, &file->f_ra,
> -							file, index);
> +			/*
> +			 * Do not do readahead for DIRECT i/o
> +			 */
> +			if (!(file->f_flags & O_DIRECT)) {
> +				struct address_space *mapping =
> +					file->f_dentry->d_inode->i_mapping;
> +				unsigned long index = kiocb->ki_pos >> PAGE_CACHE_SHIFT;
> +				unsigned long end = (kiocb->ki_pos + kiocb->ki_left)
> +					>> PAGE_CACHE_SHIFT;
> +
> +				for (; index < end; index++) {
> +					page_cache_readahead(mapping, &file->f_ra,
> +								file, index);
> +				}
>  			}
>  			kiocb->ki_retry = aio_pread;
>  		}


-- 
Suparna Bhattacharya (suparna@in.ibm.com)
Linux Technology Center
IBM Software Labs, India