Re: AIO read returns negative number for bytes read

Re: AIO read returns negative number for bytes read

[Jeff Moyer]

Hi Avi,

Avi Kivity <avi@scylladb.com> writes:

> Due to a bug in my program, I initiated a read beyond
> eof. Specifically, the file size is 13002 bytes and the read offset is
> 13312 (0x3400).
>
> I would expect such a read to return 0 bytes read, but io_getevents
> returns -310, which is suspiciously equal to (13002 - 13312).
>
> I attach a reproducer.
>
> 4.2.5-201.fc22.x86_64
>
> Are my expectations incorrect, or is this a bug in aio or xfs?

Your expectations are correct.  The bug was introduced by commit
9fe55eea7e4b4 (Fix race when checking i_size on direct i/o read).  I've
CC'd the patch author and linux-fsdevel.  I'm not sure what the right
fix is, given that the size checks were removed from the vfs to fix some
race condition.  Unfortunately, the commit message doesn't really do a
good job of explaining the race.  In order to save others time, here is
a good explanation of the problem that commit is meant to fix, along
with a reproducer:
  http://marc.info/?l=linux-fsdevel&m=138641356614458&w=2

Thanks for the great bug report, and sorry I have no solution to
proffer.

Cheers,
Jeff

Re: AIO read returns negative number for bytes read

[Avi Kivity]

On 11/16/2015 09:27 PM, Jeff Moyer wrote:
> Hi Avi,
>
> Avi Kivity <avi@scylladb.com> writes:
>
>> Due to a bug in my program, I initiated a read beyond
>> eof. Specifically, the file size is 13002 bytes and the read offset is
>> 13312 (0x3400).
>>
>> I would expect such a read to return 0 bytes read, but io_getevents
>> returns -310, which is suspiciously equal to (13002 - 13312).
>>
>> I attach a reproducer.
>>
>> 4.2.5-201.fc22.x86_64
>>
>> Are my expectations incorrect, or is this a bug in aio or xfs?
> Your expectations are correct.  The bug was introduced by commit
> 9fe55eea7e4b4 (Fix race when checking i_size on direct i/o read).  I've
> CC'd the patch author and linux-fsdevel.  I'm not sure what the right
> fix is, given that the size checks were removed from the vfs to fix some
> race condition.  Unfortunately, the commit message doesn't really do a
> good job of explaining the race.  In order to save others time, here is
> a good explanation of the problem that commit is meant to fix, along
> with a reproducer:
>    http://marc.info/?l=linux-fsdevel&m=138641356614458&w=2
>
> Thanks for the great bug report, and sorry I have no solution to
> proffer.
>

Thanks.  I will await a fix with interest.

Re: AIO read returns negative number for bytes read

[Jan Kara]

On Tue 17-11-15 14:52:19, Avi Kivity wrote:
> 
> 
> On 11/16/2015 09:27 PM, Jeff Moyer wrote:
> >Hi Avi,
> >
> >Avi Kivity <avi@scylladb.com> writes:
> >
> >>Due to a bug in my program, I initiated a read beyond
> >>eof. Specifically, the file size is 13002 bytes and the read offset is
> >>13312 (0x3400).
> >>
> >>I would expect such a read to return 0 bytes read, but io_getevents
> >>returns -310, which is suspiciously equal to (13002 - 13312).
> >>
> >>I attach a reproducer.
> >>
> >>4.2.5-201.fc22.x86_64
> >>
> >>Are my expectations incorrect, or is this a bug in aio or xfs?
> >Your expectations are correct.  The bug was introduced by commit
> >9fe55eea7e4b4 (Fix race when checking i_size on direct i/o read).  I've
> >CC'd the patch author and linux-fsdevel.  I'm not sure what the right
> >fix is, given that the size checks were removed from the vfs to fix some
> >race condition.  Unfortunately, the commit message doesn't really do a
> >good job of explaining the race.  In order to save others time, here is
> >a good explanation of the problem that commit is meant to fix, along
> >with a reproducer:
> >   http://marc.info/?l=linux-fsdevel&m=138641356614458&w=2
> >
> >Thanks for the great bug report, and sorry I have no solution to
> >proffer.
> >
> 
> Thanks.  I will await a fix with interest.

Can you please post the reproduce here as well? I couldn't easily find it
with google.

								Honza

-- 
Jan Kara <jack@suse.com>
SUSE Labs, CR

Re: AIO read returns negative number for bytes read

[Avi Kivity]

[-- Attachment #1: Type: text/plain, Size: 1575 bytes --]

On 11/19/2015 05:30 PM, Jan Kara wrote:
> On Tue 17-11-15 14:52:19, Avi Kivity wrote:
>>
>> On 11/16/2015 09:27 PM, Jeff Moyer wrote:
>>> Hi Avi,
>>>
>>> Avi Kivity <avi@scylladb.com> writes:
>>>
>>>> Due to a bug in my program, I initiated a read beyond
>>>> eof. Specifically, the file size is 13002 bytes and the read offset is
>>>> 13312 (0x3400).
>>>>
>>>> I would expect such a read to return 0 bytes read, but io_getevents
>>>> returns -310, which is suspiciously equal to (13002 - 13312).
>>>>
>>>> I attach a reproducer.
>>>>
>>>> 4.2.5-201.fc22.x86_64
>>>>
>>>> Are my expectations incorrect, or is this a bug in aio or xfs?
>>> Your expectations are correct.  The bug was introduced by commit
>>> 9fe55eea7e4b4 (Fix race when checking i_size on direct i/o read).  I've
>>> CC'd the patch author and linux-fsdevel.  I'm not sure what the right
>>> fix is, given that the size checks were removed from the vfs to fix some
>>> race condition.  Unfortunately, the commit message doesn't really do a
>>> good job of explaining the race.  In order to save others time, here is
>>> a good explanation of the problem that commit is meant to fix, along
>>> with a reproducer:
>>>    http://marc.info/?l=linux-fsdevel&m=138641356614458&w=2
>>>
>>> Thanks for the great bug report, and sorry I have no solution to
>>> proffer.
>>>
>> Thanks.  I will await a fix with interest.
> Can you please post the reproduce here as well? I couldn't easily find it
> with google.
>
>

Attached.

I am told that a simple synchronous O_DIRECT read beyond unaligned eof 
suffices as well.


[-- Attachment #2: aio_read_fails.c --]
[-- Type: text/plain, Size: 966 bytes --]

#define _GNU_SOURCE

#include <libaio.h>
#include <assert.h>
#include <stdlib.h>
#include <sys/types.h>
#include <unistd.h>
#include <stdio.h>
#include <sys/stat.h>
#include <fcntl.h>

int main(int ac, char** av) {
  int fd;
  char* buf;
  io_context_t ioc = NULL;
  int r;
  struct iocb iocb;
  struct iocb *iocbp[1];
  struct io_event ioev;

  buf = aligned_alloc(4096, 4096*4);
  assert(buf);
  r = io_setup(1, &ioc);
  assert(r == 0);
  fd = open("tmp.tmp", O_RDWR | O_CREAT | O_DIRECT, 0600);
  assert(fd >= 0);
  io_prep_pwrite(&iocb, fd, buf, 4096*4, 0);
  iocbp[0] = &iocb;
  r = io_submit(ioc, 1, iocbp);
  assert(r == 1);
  r = io_getevents(ioc, 1, 1, &ioev, NULL);
  assert(r == 1);
  assert(ioev.res == 4*4096);
  ftruncate(fd, 13002);
  io_prep_pread(&iocb, fd, buf, 8192, 13312);
  r = io_submit(ioc, 1, iocbp);
  assert(r == 1);
  r = io_getevents(ioc, 1, 1, &ioev, NULL);
  assert(r == 1);
  printf("read result: %d\n", (int)ioev.res);
  return 0;
}

Re: AIO read returns negative number for bytes read

[Jeff Moyer]

Jan Kara <jack@suse.cz> writes:

> Can you please post the reproduce here as well? I couldn't easily find it
> with google.

Here it is, Jan.

-Jeff

#define _GNU_SOURCE

#include <libaio.h>
#include <assert.h>
#include <stdlib.h>
#include <sys/types.h>
#include <unistd.h>
#include <stdio.h>
#include <sys/stat.h>
#include <fcntl.h>

int main(int ac, char** av) {
  int fd;
  char* buf;
  io_context_t ioc = NULL;
  int r;
  struct iocb iocb;
  struct iocb *iocbp[1];
  struct io_event ioev;

  buf = aligned_alloc(4096, 4096*4);
  assert(buf);
  r = io_setup(1, &ioc);
  assert(r == 0);
  fd = open("tmp.tmp", O_RDWR | O_CREAT | O_DIRECT, 0600);
  assert(fd >= 0);
  io_prep_pwrite(&iocb, fd, buf, 4096*4, 0);
  iocbp[0] = &iocb;
  r = io_submit(ioc, 1, iocbp);
  assert(r == 1);
  r = io_getevents(ioc, 1, 1, &ioev, NULL);
  assert(r == 1);
  assert(ioev.res == 4*4096);
  ftruncate(fd, 13002);
  io_prep_pread(&iocb, fd, buf, 8192, 13312);
  r = io_submit(ioc, 1, iocbp);
  assert(r == 1);
  r = io_getevents(ioc, 1, 1, &ioev, NULL);
  assert(r == 1);
  printf("read result: %d\n", (int)ioev.res);
  return 0;
}