splice() and file offsets

splice() and file offsets

[Michael Kerrisk]

Jens,

What are the semantics of splice() supposed to be with respect 
to the current file offsets of 'fd_in' and 'fd_out', and how
is the presence or absence (NULL) of 'off_in' and 'off_out'
supposed to affect things.

Using the program below, here is what I observe for 
fd_out/off_out:

1. If off_out is NULL, then 
   a) splice() changes the current file offset of fd_out.

2. If off_out is not NULL, then splice() 
   a) does not change the current file offset of fd_out, but 
   b) treats off_out as a value result parameter, returning 
      an updated offset of the file.

It is "2 a)" that surprises me.  But perhaps it's expected 
behaviour; or I'm doing something dumb in my test program.

If the test program is run without a third command-line 
argument, then 'off_out' is specified as NULL, and we see:

$ find .. | ./t_splice out
splice() returned 69632
After splice(), offset of fd is: 69632; off_out = 0
splice() returned 32768
After splice(), offset of fd is: 102400; off_out = 0
splice() returned 100000
After splice(), offset of fd is: 202400; off_out = 0
splice() returned 14688
After splice(), offset of fd is: 217088; off_out = 0
splice() returned 31990
After splice(), offset of fd is: 249078; off_out = 0
splice() returned 0
After splice(), offset of fd is: 249078; off_out = 0

If the test program is run with a third command-line 
argument, then 'off_out' is initialised to 0, and then
supplied as an argument on each splice() call, and we 
see:

$ find .. | ./t_splice out x
splice() returned 20480
After splice(), offset of fd is: 0; off_out = 20480
splice() returned 49152
After splice(), offset of fd is: 0; off_out = 69632
splice() returned 100000
After splice(), offset of fd is: 0; off_out = 169632
splice() returned 35168
After splice(), offset of fd is: 0; off_out = 204800
splice() returned 44278
After splice(), offset of fd is: 0; off_out = 249078
splice() returned 0
After splice(), offset of fd is: 0; off_out = 249078

Cheers,

Michael

/* t_splice.c */

#define _GNU_SOURCE
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <fcntl.h>
#include <assert.h>
#include <errno.h>
#include <limits.h>

#if defined(__i386__)
#define __NR_splice     313
#define __NR_tee        315
#else
#error unsupported arch
#endif

#define SPLICE_F_MOVE   (0x01)  /* move pages instead of copying */
#define SPLICE_F_NONBLOCK (0x02) /* don't block on the pipe splicing (but */
                                 /* we may still block on the fd we splice */
                                 /* from/to, of course */
#define SPLICE_F_MORE   (0x04)  /* expect more data */
#define SPLICE_F_GIFT   (0x08)  /* pages passed in are a gift */


static inline int splice(int fdin, loff_t *off_in, int fdout,
                         loff_t *off_out, size_t len, unsigned int flags)
{
    return syscall(__NR_splice, fdin, off_in, fdout, off_out, len, flags);
}

static inline int tee(int fdin, int fdout, size_t len, unsigned int flags)
{
    return syscall(__NR_tee, fdin, fdout, len, flags);
}

int
main(int argc, char *argv[])
{
    int fd;
    int slen;
    loff_t off_out;

    assert(argc >= 2);

    fd = open(argv[1], O_WRONLY | O_CREAT | O_TRUNC, 0644);
    if (fd == -1) {
        perror("open");
        exit(EXIT_FAILURE);
    }

    off_out = 0;
    for (;;) {
        slen = splice(STDIN_FILENO, NULL, fd,
                (argc > 2) ? &off_out : NULL, 100000, 0);
        if (slen < 0) {
            perror("splice");
            break;
        }
        fprintf(stderr, "splice() returned %ld\n", (long) slen);
        fprintf(stderr, "After splice(), offset of fd is: %lld; "
                "off_out = %lld\n",
                (long long) lseek(fd, 0, SEEK_CUR),
                (long long) off_out);
        if (slen == 0)
            break;
    }

    close(fd);
    exit(EXIT_SUCCESS);
}

-- 
Michael Kerrisk
maintainer of Linux man pages Sections 2, 3, 4, 5, and 7 

Want to help with man page maintenance?  
Grab the latest tarball at
ftp://ftp.win.tue.nl/pub/linux-local/manpages/, 
read the HOWTOHELP file and grep the source 
files for 'FIXME'.

Re: splice() and file offsets

[Jens Axboe]

On Mon, Jul 10 2006, Michael Kerrisk wrote:
> Jens,
> 
> What are the semantics of splice() supposed to be with respect 
> to the current file offsets of 'fd_in' and 'fd_out', and how
> is the presence or absence (NULL) of 'off_in' and 'off_out'
> supposed to affect things.
> 
> Using the program below, here is what I observe for 
> fd_out/off_out:
> 
> 1. If off_out is NULL, then 
>    a) splice() changes the current file offset of fd_out.
> 
> 2. If off_out is not NULL, then splice() 
>    a) does not change the current file offset of fd_out, but 
>    b) treats off_out as a value result parameter, returning 
>       an updated offset of the file.
> 
> It is "2 a)" that surprises me.  But perhaps it's expected 
> behaviour; or I'm doing something dumb in my test program.

Not sure why you find that surprising, that is exactly what is supposed
to happen :-)

If you don't give off_out, we use the current position. For most people,
that's probably what they want. If you are sharing the fd, that doesn't
work though. So you pass off_in/off_out as you please, and the kernel
uses those and passes the updated parameter back out so you don't have
to update it manually.

It's identical to how sendfile() works.

-- 
Jens Axboe

Re: splice() and file offsets

[Michael Kerrisk]

Jens,

> > What are the semantics of splice() supposed to be with respect 
> > to the current file offsets of 'fd_in' and 'fd_out', and how
> > is the presence or absence (NULL) of 'off_in' and 'off_out'
> > supposed to affect things.
> > 
> > Using the program below, here is what I observe for 
> > fd_out/off_out:
> > 
> > 1. If off_out is NULL, then 
> >    a) splice() changes the current file offset of fd_out.
> > 
> > 2. If off_out is not NULL, then splice() 
> >    a) does not change the current file offset of fd_out, but 
> >    b) treats off_out as a value result parameter, returning 
> >       an updated offset of the file.
> > 
> > It is "2 a)" that surprises me.  But perhaps it's expected 
> > behaviour; or I'm doing something dumb in my test program.
> 
> Not sure why you find that surprising, that is exactly what is supposed
> to happen :-)
>
> If you don't give off_out, we use the current position. For most people,
> that's probably what they want. If you are sharing the fd, that doesn't
> work though. So you pass off_in/off_out as you please, and the kernel
> uses those and passes the updated parameter back out so you don't have
> to update it manually.

I'm still not clear here.  Let me phrase my question another way:
why is it that the presence or absence of off_out affects whether
or not splice() changes the current file offset for fd_out?

> It's identical to how sendfile() works.

But it isn't: sendfile() never changes the file offset 
of its 'in_fd'.

Cheers,

Michael
-- 
Michael Kerrisk
maintainer of Linux man pages Sections 2, 3, 4, 5, and 7 

Want to help with man page maintenance?  
Grab the latest tarball at
ftp://ftp.win.tue.nl/pub/linux-local/manpages/, 
read the HOWTOHELP file and grep the source 
files for 'FIXME'.

Re: splice() and file offsets

[Jens Axboe]

On Mon, Jul 10 2006, Michael Kerrisk wrote:
> Jens,
> 
> > > What are the semantics of splice() supposed to be with respect 
> > > to the current file offsets of 'fd_in' and 'fd_out', and how
> > > is the presence or absence (NULL) of 'off_in' and 'off_out'
> > > supposed to affect things.
> > > 
> > > Using the program below, here is what I observe for 
> > > fd_out/off_out:
> > > 
> > > 1. If off_out is NULL, then 
> > >    a) splice() changes the current file offset of fd_out.
> > > 
> > > 2. If off_out is not NULL, then splice() 
> > >    a) does not change the current file offset of fd_out, but 
> > >    b) treats off_out as a value result parameter, returning 
> > >       an updated offset of the file.
> > > 
> > > It is "2 a)" that surprises me.  But perhaps it's expected 
> > > behaviour; or I'm doing something dumb in my test program.
> > 
> > Not sure why you find that surprising, that is exactly what is supposed
> > to happen :-)
> >
> > If you don't give off_out, we use the current position. For most people,
> > that's probably what they want. If you are sharing the fd, that doesn't
> > work though. So you pass off_in/off_out as you please, and the kernel
> > uses those and passes the updated parameter back out so you don't have
> > to update it manually.
> 
> I'm still not clear here.  Let me phrase my question another way:
> why is it that the presence or absence of off_out affects whether
> or not splice() changes the current file offset for fd_out?

The logic is simple - either you don't give an explicit offset, and the
current position is used and updated. Or you give an offset, and the
current position is ignored (not read, not updated).

> > It's identical to how sendfile() works.
> 
> But it isn't: sendfile() never changes the file offset 
> of its 'in_fd'.

Ehm, yes it does. Would you expect the app to do an appropriate lseek()
on every sendfile() call?

-- 
Jens Axboe

Re: splice() and file offsets

[Michael Kerrisk]

Jens,

> > I'm still not clear here.  Let me phrase my question another way:
> > why is it that the presence or absence of off_out affects whether
> > or not splice() changes the current file offset for fd_out?
> 
> The logic is simple - either you don't give an explicit offset, and the
> current position is used and updated. Or you give an offset, and the
> current position is ignored (not read, not updated).

Yes, I understand what the code is doing, but *why* do 
things this way?  (To put things another way: why not *always 
have splice() update the file offset?)  I realise there may be
some good reason for this, and if there is, it will go into the
man page!

> > > It's identical to how sendfile() works.
> > 
> > But it isn't: sendfile() never changes the file offset 
> > of its 'in_fd'.
> 
> Ehm, yes it does. Would you expect the app to do an appropriate lseek()
> on every sendfile() call?

No!  It does not!  See the sendfile.2 man page: "sendfile() 
does not modify the current file offset of in_fd."  
(You had me worried -- I just now went and *tested* 
the operation of sendfile().)  The app does not need to 
do an lseek() call because the 'offset' argument is *always* 
updated with the new "virtual" offset.  This is part of why I 
am disturbed/confused: sendfile() always updates its 'offset' 
argument and *never* changes the file offset; splice() only 
does that if its 'offset' argument is non-NULL.

Cheers,

Michael
-- 
Michael Kerrisk
maintainer of Linux man pages Sections 2, 3, 4, 5, and 7 

Want to help with man page maintenance?  
Grab the latest tarball at
ftp://ftp.win.tue.nl/pub/linux-local/manpages/, 
read the HOWTOHELP file and grep the source 
files for 'FIXME'.

Re: splice() and file offsets

[Jens Axboe]

On Mon, Jul 10 2006, Michael Kerrisk wrote:
> Jens,
> 
> > > I'm still not clear here.  Let me phrase my question another way:
> > > why is it that the presence or absence of off_out affects whether
> > > or not splice() changes the current file offset for fd_out?
> > 
> > The logic is simple - either you don't give an explicit offset, and the
> > current position is used and updated. Or you give an offset, and the
> > current position is ignored (not read, not updated).
> 
> Yes, I understand what the code is doing, but *why* do 
> things this way?  (To put things another way: why not *always 
> have splice() update the file offset?)  I realise there may be
> some good reason for this, and if there is, it will go into the
> man page!

The good reason is why update the current position? I just told the
kernel to ignore the current position and use the given offset, why
would I bother updating the current position? The whole point of
providing an offset is to ignore the current position.

I must say I cannot understand why you are confused or find this
illogical, it makes perfect sense to me.

> > > > It's identical to how sendfile() works.
> > > 
> > > But it isn't: sendfile() never changes the file offset 
> > > of its 'in_fd'.
> > 
> > Ehm, yes it does. Would you expect the app to do an appropriate lseek()
> > on every sendfile() call?
> 
> No!  It does not!  See the sendfile.2 man page: "sendfile() 
> does not modify the current file offset of in_fd."  

I didn't read the man page, I read the source. And it clearly updates
the file offset, in fact the actual sendfile portion is just a supplied
actor to the generic page cache read functions.

> (You had me worried -- I just now went and *tested* 
> the operation of sendfile().)  The app does not need to 
> do an lseek() call because the 'offset' argument is *always* 
> updated with the new "virtual" offset.  This is part of why I 
> am disturbed/confused: sendfile() always updates its 'offset' 
> argument and *never* changes the file offset; splice() only 
> does that if its 'offset' argument is non-NULL.

Maybe you didn't understand me correctly. Basically what sys_sendfile()
ends up doing is:

        if (offset_given)
                ppos = &offset_given;
        else
                ppos = &in_fd->current_position;

ppos is always updated. sendfile() behaves as I described, it updates
ppos which is _EITHER_ the supplied offset _OR_ the current file offset.
You said that sendfile() never changes in in_fd offset, which is clearly
false as it always updates it if you don't pass in an offset. If you do
pass in an offset, that and only that is changed. The lseek() comment of
course applied to the case where you _don't_ give an explicit offset and
the current position is used.

If you don't believe me, read the source and do another test app.
splice() behaves identically, as previously stated.

-- 
Jens Axboe

Re: splice() and file offsets

[Michael Kerrisk]

Jens,

> > Yes, I understand what the code is doing, but *why* do 
> > things this way?  (To put things another way: why not *always 
> > have splice() update the file offset?)  I realise there may be
> > some good reason for this, and if there is, it will go into the
> > man page!
> 
> The good reason is why update the current position? I just told the
> kernel to ignore the current position and use the given offset, why
> would I bother updating the current position? The whole point of
> providing an offset is to ignore the current position.
> 
> I must say I cannot understand why you are confused or find this
> illogical, it makes perfect sense to me.

Yes, now it's clear to me too.

[...]

> > No!  It does not!  See the sendfile.2 man page: "sendfile() 
> > does not modify the current file offset of in_fd."  
> 
> I didn't read the man page, I read the source. And it clearly updates
> the file offset, in fact the actual sendfile portion is just a supplied
> actor to the generic page cache read functions.

Doh!  I took what I "knew", re-read the sendfile.2 manual page to 
check, misread the source, and then wrote an inadequate 
test program :-{.  (The sendfile manual page is now fixed.)

> If you don't believe me, read the source and do another test app.
> splice() behaves identically, as previously stated.

Now I believe you; sorry to have wasted your time...

Cheers,

Michael
-- 
Michael Kerrisk
maintainer of Linux man pages Sections 2, 3, 4, 5, and 7 

Want to help with man page maintenance?  
Grab the latest tarball at
ftp://ftp.win.tue.nl/pub/linux-local/manpages/, 
read the HOWTOHELP file and grep the source 
files for 'FIXME'.

Re: splice() and file offsets

[Jens Axboe]

On Mon, Jul 10 2006, Michael Kerrisk wrote:
> Jens,
> 
> > > Yes, I understand what the code is doing, but *why* do 
> > > things this way?  (To put things another way: why not *always 
> > > have splice() update the file offset?)  I realise there may be
> > > some good reason for this, and if there is, it will go into the
> > > man page!
> > 
> > The good reason is why update the current position? I just told the
> > kernel to ignore the current position and use the given offset, why
> > would I bother updating the current position? The whole point of
> > providing an offset is to ignore the current position.
> > 
> > I must say I cannot understand why you are confused or find this
> > illogical, it makes perfect sense to me.
> 
> Yes, now it's clear to me too.
> 
> [...]
> 
> > > No!  It does not!  See the sendfile.2 man page: "sendfile() 
> > > does not modify the current file offset of in_fd."  
> > 
> > I didn't read the man page, I read the source. And it clearly updates
> > the file offset, in fact the actual sendfile portion is just a supplied
> > actor to the generic page cache read functions.
> 
> Doh!  I took what I "knew", re-read the sendfile.2 manual page to 
> check, misread the source, and then wrote an inadequate 
> test program :-{.  (The sendfile manual page is now fixed.)
> 
> > If you don't believe me, read the source and do another test app.
> > splice() behaves identically, as previously stated.
> 
> Now I believe you; sorry to have wasted your time...

No worries, glad we got it sorted out :-)

-- 
Jens Axboe