splice status

splice status

[Jens Axboe]

Hi,

Since a lot of splice/tee stuff has been merged, I thought I'd post a
little status report and other potentially useful info.

- splice interfaces should be stable now, I don't envision any further
  changes to the ->splice_read or ->splice_write file_operations hooks
  or the splice syscall. splice now accepts an input or output offset
  like sendfile(), so it doesn't have to rely on ->f_pos in the file
  structure.

- Ditto for the sys_tee syscall.

- sendfile() will be replaced with splice(). sys_sendfile will remain of
  course, this is only an internal thing. The current do_splice_direct()
  is a sendfile() helper. The splice branch in the block git repo has
  a patch to remove generic_file_sendfile() and all it's users by
  converting them to ->splice_read(). There's also a patch there that
  fixes up loop. The only remaining users of the file_operations
  .sendfile hook are nfds/shmem/ext2-xip/relay. That still needs doing.
  The current plan is to merge this stuff post 2.6.17.

I have a little collection of splice test tools that people may find
useful to play with this stuff. It's in a git repo here:

        git://brick.kernel.dk/data/git/splice.git

and snapshots are generated every hour on changes and can be fetched
from

        http://brick.kernel.dk/snaps/

There are tools there to test both splice and tee, a little README
explains the basic principle of them. I'd appreciate people testing and
playing with these tools, just in case we still have some bugs lurking.

Finally, known bugs:

- Some smallish splice reads are buggy. Patch is in splice branch and
  will hopefully be merged whenever Linus gets in front of his computer.

- The ->splice_pipe cache needs to be initialized to NULL on forks. Only
  affects do_splice_direct() usage, so not a problem in current kernels.
  Patch also Linus bound today.


-- 
Jens Axboe

Re: splice status - git clone failed.

[Piet Delaney]

On Thu, 2006-04-20 at 16:29 +0200, Jens Axboe wrote:
> git://brick.kernel.dk/data/git/splice.git

I just installed the latest git (20-Apr-2006) and tried
to clone your example and got an EOF. Any suggestions?
-------------------------------------------------------
/nethome/piet/src/splice$ git clone
git://brick.kernel.dk/data/git/splice.git
fatal: unexpected EOF
fetch-pack from 'git://brick.kernel.dk/data/git/splice.git' failed.
------------------------------------------------------

The shapshot is fine.

-- 
---
piet@bluelane.com

Re: splice status - git clone failed.

[Jens Axboe]

On Thu, Apr 20 2006, Piet Delaney wrote:
> On Thu, 2006-04-20 at 16:29 +0200, Jens Axboe wrote:
> > git://brick.kernel.dk/data/git/splice.git
> 
> I just installed the latest git (20-Apr-2006) and tried
> to clone your example and got an EOF. Any suggestions?
> -------------------------------------------------------
> /nethome/piet/src/splice$ git clone
> git://brick.kernel.dk/data/git/splice.git
> fatal: unexpected EOF
> fetch-pack from 'git://brick.kernel.dk/data/git/splice.git' failed.
> ------------------------------------------------------

Oh, new directory and I forgot to kick the git-daemon. It should work
now.

-- 
Jens Axboe

Re: splice status

[Christoph Hellwig]

On Thu, Apr 20, 2006 at 04:29:03PM +0200, Jens Axboe wrote:
>   converting them to ->splice_read(). There's also a patch there that
>   fixes up loop.

It actuaklly breaks loop in various setup.  You now directly call
do_generic_file_read which is just a library function for filesystems.
For example xfs or ocfs actually do need additional locking and/or other
bits before calling it.  So this absolutely has to go through a file
operation.