AIO read returns negative number for bytes read

AIO read returns negative number for bytes read

[Avi Kivity]

[-- Attachment #1: Type: text/plain, Size: 385 bytes --]

Due to a bug in my program, I initiated a read beyond eof. Specifically, 
the file size is 13002 bytes and the read offset is 13312 (0x3400).

I would expect such a read to return 0 bytes read, but io_getevents 
returns -310, which is suspiciously equal to (13002 - 13312).

I attach a reproducer.

4.2.5-201.fc22.x86_64

Are my expectations incorrect, or is this a bug in aio or xfs?

[-- Attachment #2: aio_read_fails.c --]
[-- Type: text/plain, Size: 966 bytes --]

#define _GNU_SOURCE

#include <libaio.h>
#include <assert.h>
#include <stdlib.h>
#include <sys/types.h>
#include <unistd.h>
#include <stdio.h>
#include <sys/stat.h>
#include <fcntl.h>

int main(int ac, char** av) {
  int fd;
  char* buf;
  io_context_t ioc = NULL;
  int r;
  struct iocb iocb;
  struct iocb *iocbp[1];
  struct io_event ioev;

  buf = aligned_alloc(4096, 4096*4);
  assert(buf);
  r = io_setup(1, &ioc);
  assert(r == 0);
  fd = open("tmp.tmp", O_RDWR | O_CREAT | O_DIRECT, 0600);
  assert(fd >= 0);
  io_prep_pwrite(&iocb, fd, buf, 4096*4, 0);
  iocbp[0] = &iocb;
  r = io_submit(ioc, 1, iocbp);
  assert(r == 1);
  r = io_getevents(ioc, 1, 1, &ioev, NULL);
  assert(r == 1);
  assert(ioev.res == 4*4096);
  ftruncate(fd, 13002);
  io_prep_pread(&iocb, fd, buf, 8192, 13312);
  r = io_submit(ioc, 1, iocbp);
  assert(r == 1);
  r = io_getevents(ioc, 1, 1, &ioev, NULL);
  assert(r == 1);
  printf("read result: %d\n", (int)ioev.res);
  return 0;
}

[-- Attachment #3: Type: text/plain, Size: 121 bytes --]

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: AIO read returns negative number for bytes read

[Brian Foster]

On Sun, Nov 15, 2015 at 03:08:13PM +0200, Avi Kivity wrote:
> Due to a bug in my program, I initiated a read beyond eof. Specifically, the
> file size is 13002 bytes and the read offset is 13312 (0x3400).
> 
> I would expect such a read to return 0 bytes read, but io_getevents returns
> -310, which is suspiciously equal to (13002 - 13312).
> 
> I attach a reproducer.
> 
> 4.2.5-201.fc22.x86_64
> 
> Are my expectations incorrect, or is this a bug in aio or xfs?

FWIW, I added some printk()'s and reproduced. This looks like a dio
issue to me (a sync dio read reproduces just the same as aio). See the
short read check in dio_complete():


	...
        /* Check for short read case */
        if ((dio->rw == READ) && ((offset + transferred) > dio->i_size))
                transferred = dio->i_size - offset;
	...

So if offset starts beyond i_size, we set transferred to a negative
value. I suppose we could check for offset >= i_size here independently,
but I'm not necessarily sure something earlier doesn't need to change
(e.g., I see dio->result = 3072 in this case and I'm not familiar enough
with core dio to know if that's expected).

Brian

> #define _GNU_SOURCE
> 
> #include <libaio.h>
> #include <assert.h>
> #include <stdlib.h>
> #include <sys/types.h>
> #include <unistd.h>
> #include <stdio.h>
> #include <sys/stat.h>
> #include <fcntl.h>
> 
> int main(int ac, char** av) {
>   int fd;
>   char* buf;
>   io_context_t ioc = NULL;
>   int r;
>   struct iocb iocb;
>   struct iocb *iocbp[1];
>   struct io_event ioev;
> 
>   buf = aligned_alloc(4096, 4096*4);
>   assert(buf);
>   r = io_setup(1, &ioc);
>   assert(r == 0);
>   fd = open("tmp.tmp", O_RDWR | O_CREAT | O_DIRECT, 0600);
>   assert(fd >= 0);
>   io_prep_pwrite(&iocb, fd, buf, 4096*4, 0);
>   iocbp[0] = &iocb;
>   r = io_submit(ioc, 1, iocbp);
>   assert(r == 1);
>   r = io_getevents(ioc, 1, 1, &ioev, NULL);
>   assert(r == 1);
>   assert(ioev.res == 4*4096);
>   ftruncate(fd, 13002);
>   io_prep_pread(&iocb, fd, buf, 8192, 13312);
>   r = io_submit(ioc, 1, iocbp);
>   assert(r == 1);
>   r = io_getevents(ioc, 1, 1, &ioev, NULL);
>   assert(r == 1);
>   printf("read result: %d\n", (int)ioev.res);
>   return 0;
> }

> _______________________________________________
> xfs mailing list
> xfs@oss.sgi.com
> http://oss.sgi.com/mailman/listinfo/xfs

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: AIO read returns negative number for bytes read

[Alireza Haghdoost]

On Sun, Nov 15, 2015 at 7:08 AM, Avi Kivity <avi@scylladb.com> wrote:
> Due to a bug in my program, I initiated a read beyond eof. Specifically, the
> file size is 13002 bytes and the read offset is 13312 (0x3400).
>
> I would expect such a read to return 0 bytes read, but io_getevents returns
> -310, which is suspiciously equal to (13002 - 13312).
>
> I attach a reproducer.
>
> 4.2.5-201.fc22.x86_64
>
> Are my expectations incorrect, or is this a bug in aio or xfs?

I think it is not a bug. This post might be helpful:
http://marc.info/?l=linux-aio&m=142315449930935&w=2

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: AIO read returns negative number for bytes read

[Avi Kivity]

On 11/16/2015 05:19 PM, Alireza Haghdoost wrote:
> On Sun, Nov 15, 2015 at 7:08 AM, Avi Kivity <avi@scylladb.com> wrote:
>> Due to a bug in my program, I initiated a read beyond eof. Specifically, the
>> file size is 13002 bytes and the read offset is 13312 (0x3400).
>>
>> I would expect such a read to return 0 bytes read, but io_getevents returns
>> -310, which is suspiciously equal to (13002 - 13312).
>>
>> I attach a reproducer.
>>
>> 4.2.5-201.fc22.x86_64
>>
>> Are my expectations incorrect, or is this a bug in aio or xfs?
> I think it is not a bug. This post might be helpful:
> http://marc.info/?l=linux-aio&m=142315449930935&w=2

The post indicates that it is a bug.  A negative value indicates an 
error, but there is no error 310.

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: AIO read returns negative number for bytes read

[Jeff Moyer]

Hi Avi,

Avi Kivity <avi@scylladb.com> writes:

> Due to a bug in my program, I initiated a read beyond
> eof. Specifically, the file size is 13002 bytes and the read offset is
> 13312 (0x3400).
>
> I would expect such a read to return 0 bytes read, but io_getevents
> returns -310, which is suspiciously equal to (13002 - 13312).
>
> I attach a reproducer.
>
> 4.2.5-201.fc22.x86_64
>
> Are my expectations incorrect, or is this a bug in aio or xfs?

Your expectations are correct.  The bug was introduced by commit
9fe55eea7e4b4 (Fix race when checking i_size on direct i/o read).  I've
CC'd the patch author and linux-fsdevel.  I'm not sure what the right
fix is, given that the size checks were removed from the vfs to fix some
race condition.  Unfortunately, the commit message doesn't really do a
good job of explaining the race.  In order to save others time, here is
a good explanation of the problem that commit is meant to fix, along
with a reproducer:
  http://marc.info/?l=linux-fsdevel&m=138641356614458&w=2

Thanks for the great bug report, and sorry I have no solution to
proffer.

Cheers,
Jeff

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: AIO read returns negative number for bytes read

[Avi Kivity]

On 11/16/2015 09:27 PM, Jeff Moyer wrote:
> Hi Avi,
>
> Avi Kivity <avi@scylladb.com> writes:
>
>> Due to a bug in my program, I initiated a read beyond
>> eof. Specifically, the file size is 13002 bytes and the read offset is
>> 13312 (0x3400).
>>
>> I would expect such a read to return 0 bytes read, but io_getevents
>> returns -310, which is suspiciously equal to (13002 - 13312).
>>
>> I attach a reproducer.
>>
>> 4.2.5-201.fc22.x86_64
>>
>> Are my expectations incorrect, or is this a bug in aio or xfs?
> Your expectations are correct.  The bug was introduced by commit
> 9fe55eea7e4b4 (Fix race when checking i_size on direct i/o read).  I've
> CC'd the patch author and linux-fsdevel.  I'm not sure what the right
> fix is, given that the size checks were removed from the vfs to fix some
> race condition.  Unfortunately, the commit message doesn't really do a
> good job of explaining the race.  In order to save others time, here is
> a good explanation of the problem that commit is meant to fix, along
> with a reproducer:
>    http://marc.info/?l=linux-fsdevel&m=138641356614458&w=2
>
> Thanks for the great bug report, and sorry I have no solution to
> proffer.
>

Thanks.  I will await a fix with interest.


_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: AIO read returns negative number for bytes read

[Jan Kara]

On Tue 17-11-15 14:52:19, Avi Kivity wrote:
> 
> 
> On 11/16/2015 09:27 PM, Jeff Moyer wrote:
> >Hi Avi,
> >
> >Avi Kivity <avi@scylladb.com> writes:
> >
> >>Due to a bug in my program, I initiated a read beyond
> >>eof. Specifically, the file size is 13002 bytes and the read offset is
> >>13312 (0x3400).
> >>
> >>I would expect such a read to return 0 bytes read, but io_getevents
> >>returns -310, which is suspiciously equal to (13002 - 13312).
> >>
> >>I attach a reproducer.
> >>
> >>4.2.5-201.fc22.x86_64
> >>
> >>Are my expectations incorrect, or is this a bug in aio or xfs?
> >Your expectations are correct.  The bug was introduced by commit
> >9fe55eea7e4b4 (Fix race when checking i_size on direct i/o read).  I've
> >CC'd the patch author and linux-fsdevel.  I'm not sure what the right
> >fix is, given that the size checks were removed from the vfs to fix some
> >race condition.  Unfortunately, the commit message doesn't really do a
> >good job of explaining the race.  In order to save others time, here is
> >a good explanation of the problem that commit is meant to fix, along
> >with a reproducer:
> >   http://marc.info/?l=linux-fsdevel&m=138641356614458&w=2
> >
> >Thanks for the great bug report, and sorry I have no solution to
> >proffer.
> >
> 
> Thanks.  I will await a fix with interest.

Can you please post the reproduce here as well? I couldn't easily find it
with google.

								Honza

-- 
Jan Kara <jack@suse.com>
SUSE Labs, CR

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: AIO read returns negative number for bytes read

[Avi Kivity]

[-- Attachment #1: Type: text/plain, Size: 1575 bytes --]

On 11/19/2015 05:30 PM, Jan Kara wrote:
> On Tue 17-11-15 14:52:19, Avi Kivity wrote:
>>
>> On 11/16/2015 09:27 PM, Jeff Moyer wrote:
>>> Hi Avi,
>>>
>>> Avi Kivity <avi@scylladb.com> writes:
>>>
>>>> Due to a bug in my program, I initiated a read beyond
>>>> eof. Specifically, the file size is 13002 bytes and the read offset is
>>>> 13312 (0x3400).
>>>>
>>>> I would expect such a read to return 0 bytes read, but io_getevents
>>>> returns -310, which is suspiciously equal to (13002 - 13312).
>>>>
>>>> I attach a reproducer.
>>>>
>>>> 4.2.5-201.fc22.x86_64
>>>>
>>>> Are my expectations incorrect, or is this a bug in aio or xfs?
>>> Your expectations are correct.  The bug was introduced by commit
>>> 9fe55eea7e4b4 (Fix race when checking i_size on direct i/o read).  I've
>>> CC'd the patch author and linux-fsdevel.  I'm not sure what the right
>>> fix is, given that the size checks were removed from the vfs to fix some
>>> race condition.  Unfortunately, the commit message doesn't really do a
>>> good job of explaining the race.  In order to save others time, here is
>>> a good explanation of the problem that commit is meant to fix, along
>>> with a reproducer:
>>>    http://marc.info/?l=linux-fsdevel&m=138641356614458&w=2
>>>
>>> Thanks for the great bug report, and sorry I have no solution to
>>> proffer.
>>>
>> Thanks.  I will await a fix with interest.
> Can you please post the reproduce here as well? I couldn't easily find it
> with google.
>
>

Attached.

I am told that a simple synchronous O_DIRECT read beyond unaligned eof 
suffices as well.


[-- Attachment #2: aio_read_fails.c --]
[-- Type: text/plain, Size: 966 bytes --]

#define _GNU_SOURCE

#include <libaio.h>
#include <assert.h>
#include <stdlib.h>
#include <sys/types.h>
#include <unistd.h>
#include <stdio.h>
#include <sys/stat.h>
#include <fcntl.h>

int main(int ac, char** av) {
  int fd;
  char* buf;
  io_context_t ioc = NULL;
  int r;
  struct iocb iocb;
  struct iocb *iocbp[1];
  struct io_event ioev;

  buf = aligned_alloc(4096, 4096*4);
  assert(buf);
  r = io_setup(1, &ioc);
  assert(r == 0);
  fd = open("tmp.tmp", O_RDWR | O_CREAT | O_DIRECT, 0600);
  assert(fd >= 0);
  io_prep_pwrite(&iocb, fd, buf, 4096*4, 0);
  iocbp[0] = &iocb;
  r = io_submit(ioc, 1, iocbp);
  assert(r == 1);
  r = io_getevents(ioc, 1, 1, &ioev, NULL);
  assert(r == 1);
  assert(ioev.res == 4*4096);
  ftruncate(fd, 13002);
  io_prep_pread(&iocb, fd, buf, 8192, 13312);
  r = io_submit(ioc, 1, iocbp);
  assert(r == 1);
  r = io_getevents(ioc, 1, 1, &ioev, NULL);
  assert(r == 1);
  printf("read result: %d\n", (int)ioev.res);
  return 0;
}

[-- Attachment #3: Type: text/plain, Size: 121 bytes --]

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: AIO read returns negative number for bytes read

[Jeff Moyer]

Jan Kara <jack@suse.cz> writes:

> Can you please post the reproduce here as well? I couldn't easily find it
> with google.

Here it is, Jan.

-Jeff

#define _GNU_SOURCE

#include <libaio.h>
#include <assert.h>
#include <stdlib.h>
#include <sys/types.h>
#include <unistd.h>
#include <stdio.h>
#include <sys/stat.h>
#include <fcntl.h>

int main(int ac, char** av) {
  int fd;
  char* buf;
  io_context_t ioc = NULL;
  int r;
  struct iocb iocb;
  struct iocb *iocbp[1];
  struct io_event ioev;

  buf = aligned_alloc(4096, 4096*4);
  assert(buf);
  r = io_setup(1, &ioc);
  assert(r == 0);
  fd = open("tmp.tmp", O_RDWR | O_CREAT | O_DIRECT, 0600);
  assert(fd >= 0);
  io_prep_pwrite(&iocb, fd, buf, 4096*4, 0);
  iocbp[0] = &iocb;
  r = io_submit(ioc, 1, iocbp);
  assert(r == 1);
  r = io_getevents(ioc, 1, 1, &ioev, NULL);
  assert(r == 1);
  assert(ioev.res == 4*4096);
  ftruncate(fd, 13002);
  io_prep_pread(&iocb, fd, buf, 8192, 13312);
  r = io_submit(ioc, 1, iocbp);
  assert(r == 1);
  r = io_getevents(ioc, 1, 1, &ioev, NULL);
  assert(r == 1);
  printf("read result: %d\n", (int)ioev.res);
  return 0;
}

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs