Re: A problem about DIRECT IO on ext3

[Jens Axboe <axboe@suse.de>]

On Mon, Oct 17 2005, li nux wrote:
> 
> 
> --- Jens Axboe <axboe@suse.de> wrote:
> 
> > On Mon, Aug 29 2005, Erik Mouw wrote:
> > > There are four prerequisites for direct IO:
> > > - the file needs to be opened with O_DIRECT
> > > - the buffer needs to be page aligned (hint: use
> > getpagesize() instead
> > >   of assuming that a page is 4k
> > > - reads and writes need to happen *in* multiples
> > of the soft block size
> > > - reads and writes need to happen *at* multiples
> > of the soft block size
> > 
> > Actually, the buffer only needs to be hard block
> > size aligned, same goes
> > for the chunk size used for reads/writes.
> > 
> > -- 
> > Jens Axboe
> > 
> On 2.4 the open call succeeds with O_DIRECT
> but read returns -EINVAL for any block size (512, 1024
> ..16384)
> 
> open("/tmp/midstress_idx10",
> O_RDWR|O_CREAT|O_DIRECT|O_LARGEFILE, 01001101270) = 4
> read(3, 0xbfffdc40, 16384) = -1 EINVAL (Invalid
> argument)
> 
> how to correct this problem ?

See your buffer address, it's not aligned. You need to align that as
well. This is needed because the hardware will dma directly to the user
buffer, and to be on the safe side we require the same alignment as the
block layer will normally generate for file system io.

So in short, just align your read buffer to the same as your block size
and you will be fine. Example:

#define BS      (4096)
#define MASK    (BS - 1)
#define ALIGN(buf)      (((unsigned long) (buf) + MASK) & ~(MASK))

char *ptr = malloc(BS + MASK);
char *buf = (char *) ALIGN(ptr);

read(fd, buf, BS);

-- 
Jens Axboe