Re: lseek gets bad offset from nfs client with ganesha/gluster which supports SEEK

[Anna Schumaker <schumaker.anna@gmail.com>]

On Tue, 2018-09-11 at 22:47 +0800, Kinglong Mee wrote:
> On 2018/9/11 20:57, Trond Myklebust wrote:
> > On Tue, 2018-09-11 at 20:29 +0800, Kinglong Mee wrote:
> > > The latest ganesha/gluster supports seek according to,
> > > 
> > > 
> > 
> > https://tools.ietf.org/html/draft-ietf-nfsv4-minorversion2-41#section-15.11
> > > 
> > >    From the given sa_offset, find the next data_content4 of type
> > > sa_what
> > >    in the file.  If the server can not find a corresponding sa_what,
> > >    then the status will still be NFS4_OK, but sr_eof would be
> > > TRUE.  If
> > >    the server can find the sa_what, then the sr_offset is the start
> > > of
> > >    that content.  If the sa_offset is beyond the end of the file,
> > > then
> > >    SEEK MUST return NFS4ERR_NXIO.
> > > 
> > > For a file's filemap as,
> > > 
> > > Part    1: HOLE 0x0000000000000000 ---> 0x0000000000600000
> > > Part    2: DATA 0x0000000000600000 ---> 0x0000000000700000
> > > Part    3: HOLE 0x0000000000700000 ---> 0x0000000001000000>>
> > > SEEK(0x700000, SEEK_DATA) gets result (sr_eof:1, sr_offset:0x70000)
> > > from ganesha/gluster;
> > > SEEK(0x700000, SEEK_HOLE) gets result (sr_eof:0, sr_offset:0x70000)
> > > from ganesha/gluster.
> > > 
> > > If an application depends the lseek result for data searching, it may
> > > enter infinite loop.
> > > 
> > >         while (1) {
> > >                 next_pos = lseek(fd, cur_pos, seek_type);
> > >                 if (seek_type == SEEK_DATA) {
> > >                         seek_type = SEEK_HOLE;
> > >                 } else {
> > >                         seek_type = SEEK_DATA;
> > >                 }
> > > 
> > >                 if (next_pos == -1) {
> > >                         return ;
> > > 
> > >                 cur_pos = next_pos;
> > >         }
> > > 
> > > The lseek syscall always gets 0x70000 from nfs client for those two
> > > cases, 
> > > but, if underlying filesystem is ext4/f2fs, or the nfs server is
> > > knfsd,
> > > the lseek(0x700000, SEEK_DATA) gets ENXIO.
> > > 
> > > I wanna to know,
> > > should I fix the ganesha/gluster as knfsd return ENXIO for the first
> > > case?
> > > or should I fix the nfs client to return ENXIO for the first case?
> > > 
> > 
> > It that test correct? The fallback implementation of SEEK_DATA assumes
> > that the entire file is data, so lseek(SEEK_DATA) on any offset that is
> > <= eof will be a no-op. The fallback implementation of SEEK_HOLE
> > assumes that the first hole is at eof.
> 
> I think that's non-nfsv4.2's logical.
> 
> > 
> > IOW: unless the initial value for cur_pos is > eof, it looks to me as
> > if the above test will loop infinitely given any filesystem that
> > doesn't implement native support for SEEK_DATA/SEEK_HOLE.
> > 
> 
> No, if underlying filesystem is ext4/f2fs, or the nfs server is knfsd,
> the last lseek syscall always return ENXIO no matter the cur_pos is > eof or
> not.
> 
> A file ends with a hole as,
> Part   22: DATA 0x0000000006a00000 ---> 0x0000000006afffff
> Part   23: HOLE 0x0000000006b00000 ---> 0x000000000c7fffff
> 
> # stat testfile
>   File: testfile
>   Size: 209715200       Blocks: 22640      IO Block: 4096   regular file
> Device: 807h/2055d      Inode: 843122      Links: 2
> Access: (0600/-rw-------)  Uid: (    0/    root)   Gid: (    0/    root)
> Access: 2018-09-11 20:01:41.881227061 +0800
> Modify: 2018-09-11 20:01:41.976478311 +0800
> Change: 2018-09-11 20:01:41.976478311 +0800
>  Birth: -
> 
> # strace filemap testfile
> ... ...
> lseek(3, 111149056, SEEK_HOLE)          = 112197632
> lseek(3, 112197632, SEEK_DATA)          = -1 ENXIO (No such device or address)
> 
> Right now, when knfsd gets the ENXIO error, it returns the error to client
> directly,
> and return to syscall.
> But, ganesha set the sr_eof to true and return NFS4_OK to client as RFC
> description,
> nfs client skips the sr_eof and return a bad offset to syscall.

Would it make more sense to change Knfsd instead of the client?  I think I was
trying to keep things simple when I wrote the code, so I just passed the result
of the lseek system call back to the client.

Anna
> 
> thanks,
> Kinglong Mee