* Re: [PATCH 0/3] vmsplice: make vmsplice a trivial wrapper for preadv2/pwritev2
From: Jakub Kicinski @ 2026-06-03 18:12 UTC (permalink / raw)
To: Linus Torvalds
Cc: Andy Lutomirski, Askar Safin, akpm, axboe, brauner, david,
dhowells, hch, jack, linux-api, linux-fsdevel, linux-kernel,
linux-mm, miklos, netdev, patches, pfalcato, viro, willy
In-Reply-To: <CAHk-=wizkDXRut5xLXRF-CVUVYMaZ5AOexxeghOAoXPb4yAvQg@mail.gmail.com>
On Tue, 2 Jun 2026 21:20:13 -0700 Linus Torvalds wrote:
> They were all in the networking and crypto code that just didn't deal
> with shared data correctly.
>
> So in that sense, it's a bit sad to discuss castrating splice.
+1 IMVHO the networking bugs where people just not knowing what they
were doing. Presumably AI has scrounged all the occurrences of that
bug by now. I'd also hate to render splice optimizations moot based
on those bugs.
^ permalink raw reply
* Re: [PATCH 0/3] vmsplice: make vmsplice a trivial wrapper for preadv2/pwritev2
From: Jakub Kicinski @ 2026-06-03 18:14 UTC (permalink / raw)
To: Pedro Falcato
Cc: Andy Lutomirski, Linus Torvalds, Askar Safin, akpm, axboe,
brauner, david, dhowells, hch, jack, linux-api, linux-fsdevel,
linux-kernel, linux-mm, miklos, netdev, patches, viro, willy
In-Reply-To: <aiAREqlHK1llOw_y@pedro-suse.lan>
On Wed, 3 Jun 2026 12:43:54 +0100 Pedro Falcato wrote:
> > Am I understanding correctly that this will completely break zerocopy
> > sendfile? sendfile is, internally, splice-to-a-secret-per-task-pipe
> > and then splice to the socket. How much to people care? These days,
> > a lot of high-bandwidth network senders are sending encrypted data,
> > which is not zerocopy frompagecache. But there are surely some users
>
> You can do zerocopy from the page cache, even with TLS on top, by having
> your (fancy) NIC do TLS offloading for you. See https://people.freebsd.org/~gallatin/talks/euro2019-ktls.pdf.
> Linux works similarly. Slide 26 is particularly interesting.
> (No KTLS I assume is using simple sendmsg()'s from user memory, SW TLS
> and NIC KTLS are both sendfile(), per the slides)
FTR this datapoint should come with the caveat that kTLS _offload_ does
not support TLS 1.3 today. So how much that configuration is used in
practice is unclear.
^ permalink raw reply
* Re: [PATCH 0/3] vmsplice: make vmsplice a trivial wrapper for preadv2/pwritev2
From: Linus Torvalds @ 2026-06-03 18:28 UTC (permalink / raw)
To: Andy Lutomirski
Cc: Askar Safin, akpm, axboe, brauner, david, dhowells, hch, jack,
linux-api, linux-fsdevel, linux-kernel, linux-mm, miklos, netdev,
patches, pfalcato, viro, willy
In-Reply-To: <CALCETrXzxubt4eWue3+wv7Fq9C2m7uu6bWPstqFh6Mo57bPwQQ@mail.gmail.com>
On Wed, 3 Jun 2026 at 11:10, Andy Lutomirski <luto@amacapital.net> wrote:
>
> So maybe we should make sure that, if we go down the route of
> disabling all the splice magic, that we leave an API, maybe the
> existing sendfile or maybe something else, that does an optimized copy
> from one fd to another and that is at least capable of sending from a
> file to the network with at most one CPU-side copy.
Why?
That is *LITERALLY* the attack surface - and the complexity - that we
should be removing.
sendfile() was a mistake. It is literally the "file->socket" thing
that has been buggy.
I absolutely refuse to get rid of splice code but keep the buggy sh*t
cases that caused all the problems in the first place.
Because *THAT* would just be completely insane and pointless.
> Even if we’re just doing that, I continue to find it strange that we
> require that a pipe be involved. What’s so special about pipes
Again: it was never splice or the pipe that was the problem. Stop
barking up the wrong tree.
It was "file data to socket" that was the truly horrendous issue.
That said, to explain the pipe: The reason for the pipe is to act as
the kernel-side buffer.
Now, these days we have much more capable iov_iter interfaces than we
used to, and in that sense the "pipe as a buffer" is certainly not the
obvious choice now.
But even then you need to have a *handle* to the buffers for the
general case, and that's what the pipe fd ends up then still
effectively being.
It was also done to avoid the M:N translation problem, because people
wanted to do zero-copy between other things than just "file ->
socket".
But again: we're ABNSOLUTELY NOT keeping that "file -> socket" thing
and getting rid of splice. That's literally keeping the bath-water
and throwing out the baby.
Splice is the *good* part (well, relatively - splice is bad too).
ile->socket needs to DIE IN A FIRE considering the security problems it has had.
I hope Jakub is right that the problems have been all fixed, and this
is all theoretical, but having seen just *how* many there were, I'm a
bit sceptical.
Because if people think splice is complicated, you haven't looked at
the skb rules. They are completely arbitrary and complex and spread
all over the tree.
Linus
^ permalink raw reply
* Re: [PATCH 0/3] vmsplice: make vmsplice a trivial wrapper for preadv2/pwritev2
From: David Howells @ 2026-06-03 19:22 UTC (permalink / raw)
To: Linus Torvalds
Cc: dhowells, Andy Lutomirski, Askar Safin, akpm, axboe, brauner,
david, hch, jack, linux-api, linux-fsdevel, linux-kernel,
linux-mm, miklos, netdev, patches, pfalcato, viro, willy
In-Reply-To: <CAHk-=wiEwSjfbjfO74xu=UmkkdHXkJg5QNQ8pP-3iYmunmeV9g@mail.gmail.com>
Linus Torvalds <torvalds@linux-foundation.org> wrote:
> Because if people think splice is complicated, you haven't looked at
> the skb rules. They are completely arbitrary and complex and spread
> all over the tree.
Yeah - I fell foul of the net loopback driver just reflecting the outgoing
packet back, complete with all the original spliced bufferage. I was
wondering if the loopback driver needs to look at the skbuff, see if it has
zerocopy elements of some sort and, if so, copy it (or drop it if ENOMEM).
David
^ permalink raw reply
* Re: [PATCH 0/3] vmsplice: make vmsplice a trivial wrapper for preadv2/pwritev2
From: David Howells @ 2026-06-03 19:24 UTC (permalink / raw)
To: Linus Torvalds
Cc: dhowells, Matthew Wilcox, Andy Lutomirski, Askar Safin,
linux-fsdevel, Christian Brauner, Alexander Viro, Jan Kara,
linux-kernel, linux-mm, linux-api, netdev, Jens Axboe,
Christoph Hellwig, Andrew Morton, David Hildenbrand,
Pedro Falcato, Miklos Szeredi, patches
In-Reply-To: <CAHk-=wiFuud0Nn3B9YpTWyQja08TeXVk2AB-aAkmVXyigOagbQ@mail.gmail.com>
Linus Torvalds <torvalds@linux-foundation.org> wrote:
> Well, since it pretty much is what I suggested a few years ago, I
> certainly won't NAK it.
I've been wanting to get rid of vmsplice for a while, so I'm in favour of this
too.
David
^ permalink raw reply
* Re: [PATCH 0/3] vmsplice: make vmsplice a trivial wrapper for preadv2/pwritev2
From: Linus Torvalds @ 2026-06-03 19:59 UTC (permalink / raw)
To: Andy Lutomirski
Cc: Askar Safin, akpm, axboe, brauner, david, dhowells, hch, jack,
linux-api, linux-fsdevel, linux-kernel, linux-mm, miklos, netdev,
patches, pfalcato, viro, willy
In-Reply-To: <CAHk-=wiEwSjfbjfO74xu=UmkkdHXkJg5QNQ8pP-3iYmunmeV9g@mail.gmail.com>
On Wed, 3 Jun 2026 at 11:28, Linus Torvalds
<torvalds@linux-foundation.org> wrote:
>
> But even then you need to have a *handle* to the buffers for the
> general case, and that's what the pipe fd ends up then still
> effectively being.
Again: for sendfile, you don't need the handle, because you can just
"read the file data again".
But the the handle is needed for any buffering that can't do that -
iow pretty much *any* other case than a file-backed source.
So the original use-cases included things like copying media data from
a TV capture card to a GPU for outputting in a window.
There it's actually the intermediate buffer that is the important
thing, and it needs to have a lifetime that is independent of the
system call itself, because the system call may be interrupted by
signals etc, and you can't just "read the data again" when you
restart.
So the whole idea with splice() is that you have an input, an output,
and a stateful buffer between the two that has a lifetime.
Having just a iov_iter isn't enough - even with the current much more
capable iov_iter we have now (compared to when splice came to be: two
decades ago when the modern iov_iter didn't even exist). You have to
have that notion of a buffer with a lifetime.
(iov_iter came a couple of years later, but it then took many many
years for it to become the powerful thing it is today where you can
put almost arbitrary data into it - it started as purely a user space
iovec iterator, all the bvec/kvec etc stuff that you need for IO
buffering came a decade later)
So there's historical reasons for the use of pipes, but there really
is a very fundamental reason for it too: wanting to *generic* data
transfer between two points, not sendfile.
It's worth noticing that in the generic case, zero-copy isn't really
even an issue.
When you think operations like "splice TV capture input to a pipe",
you typically need to allocate the pages that you then DMA into
*anyway*, and you'd just put those pages into the pipe. And the facty
that you can then just take the data directly from those pages when
you splice from the pipe to whatever GPU engine that does the decoding
is kind of secondary.
So again: the big deal with splice() and the pipe isn't really about
zero-copy. It's the in-kernel buffers where the drivers control the
allocation and you don't have some "user space allocates memory, then
kernel looks that allocation up and uses it" model.
Having less copies is kind of incidental. It *might* happen just
because it's natural when some streaming device just gives it data
away and doesn't care after the fact.
The problem with splicing from a file has been exactly the fact that
it's *not* streaming data, and the filesystem zero-copy case gave
direct access to the long-term cache.
Which is undoubtedly good for performance. But it fundamentally
*requires* that the sink is trustworthy. Which has been problematic.
That's why sendfile() is bad. Not because splice itself is a bad
concept, but because you have to have that absolute trust across
components.
Linus
^ permalink raw reply
* Re: [PATCH 2/3] vmsplice: make vmsplice a trivial wrapper for preadv2/pwritev2
From: Stefan Metzmacher @ 2026-06-03 20:56 UTC (permalink / raw)
To: Askar Safin, linux-fsdevel, Christian Brauner, Alexander Viro,
Jan Kara
Cc: linux-kernel, linux-mm, linux-api, netdev, Linus Torvalds,
Matthew Wilcox, Jens Axboe, Christoph Hellwig, David Howells,
Andrew Morton, David Hildenbrand, Pedro Falcato, Miklos Szeredi,
patches
In-Reply-To: <20260531010107.1953702-3-safinaskar@gmail.com>
Hi Askar,
> diff --git a/include/linux/syscalls.h b/include/linux/syscalls.h
> index f5639d5ac331..a86a88207956 100644
> --- a/include/linux/syscalls.h
> +++ b/include/linux/syscalls.h
> @@ -514,8 +514,8 @@ asmlinkage long sys_ppoll_time32(struct pollfd __user *, unsigned int,
> struct old_timespec32 __user *, const sigset_t __user *,
> size_t);
> asmlinkage long sys_signalfd4(int ufd, sigset_t __user *user_mask, size_t sizemask, int flags);
> -asmlinkage long sys_vmsplice(int fd, const struct iovec __user *iov,
> - unsigned long nr_segs, unsigned int flags);
> +asmlinkage long sys_vmsplice(unsigned long fd, const struct iovec __user *vec,
> + unsigned long vlen, unsigned int flags);
Why is 'int fd' changed to 'unsigned long fd'?
Should that be its own commit if the change is desired?
metze
^ permalink raw reply
* Re: [PATCH 2/3] vmsplice: make vmsplice a trivial wrapper for preadv2/pwritev2
From: Askar Safin @ 2026-06-03 21:17 UTC (permalink / raw)
To: metze
Cc: akpm, axboe, brauner, david, dhowells, hch, jack, linux-api,
linux-fsdevel, linux-kernel, linux-mm, miklos, netdev, patches,
pfalcato, safinaskar, torvalds, viro, willy
In-Reply-To: <f919874c-065e-48be-ac5b-300c4ab86d4e@samba.org>
Stefan Metzmacher <metze@samba.org>:
> Why is 'int fd' changed to 'unsigned long fd'?
Because preadv2 and pwritev2 take "unsigned long". I want vmsplice
to be as similar as possible to preadv2 and pwritev2.
> Should that be its own commit if the change is desired?
Yes, possibly. But this patchset already got to next.
--
Askar Safin
^ permalink raw reply
* Re: [PATCH 0/3] vmsplice: make vmsplice a trivial wrapper for preadv2/pwritev2
From: Andy Lutomirski @ 2026-06-03 21:31 UTC (permalink / raw)
To: Linus Torvalds
Cc: Askar Safin, akpm, axboe, brauner, david, dhowells, hch, jack,
linux-api, linux-fsdevel, linux-kernel, linux-mm, miklos, netdev,
patches, pfalcato, viro, willy
In-Reply-To: <CAHk-=wiEwSjfbjfO74xu=UmkkdHXkJg5QNQ8pP-3iYmunmeV9g@mail.gmail.com>
On Wed, Jun 3, 2026 at 11:29 AM Linus Torvalds
<torvalds@linux-foundation.org> wrote:
>
> On Wed, 3 Jun 2026 at 11:10, Andy Lutomirski <luto@amacapital.net> wrote:
> >
> > So maybe we should make sure that, if we go down the route of
> > disabling all the splice magic, that we leave an API, maybe the
> > existing sendfile or maybe something else, that does an optimized copy
> > from one fd to another and that is at least capable of sending from a
> > file to the network with at most one CPU-side copy.
>
> Why?
>
> That is *LITERALLY* the attack surface - and the complexity - that we
> should be removing.
I think I buried the lede too much and you're arguing against what I
was trying not to say.
Maybe we should keep an API that does an optimized copy, from one fd
to another, that can send from a file to the network with at most ONE
cpu-side copy. Not aiming for zero like sendfile / splice. Aiming
for one.
If sendfile and splice get completely deoptimized (which I think makes
a considerable amount of sense), then I think that, as you said,
there's a risk that the most efficient way to send the contents of a
file to the network is to read it into user memory and then send it,
which is *two* copies to get it from pagecache to the outgoing socket
buffer. But I think that just one copy can be done with essentially
no funny business.
copy_splice_read is conceptually not terrible at all -- it allocates
memory and copies from page cache. But splice_to_socket involves
MSG_SPLACE_PAGES, which I think is a part of the mess that you
dislike. And the path where one does copy_splice_read and then
splice_to_socket has to be a bit complex because of tee and (I think)
because splice_to_socket cannot assume that the incoming data is just
ordinary unshared buffers.
What I'm suggesting is that, at least for network families/protocols
that care to support such a thing, there could be a slightly tedious
but otherwise utterly boring path to *copy* from pagecache to socket
buffers. So, once the copy is done, the skbs would be ordinary skbs,
exactly as if the user had called plain send(), and nothing downstream
(the network drivers, crazy crypto code, etc) would ever see the
difference.
I don't think I'm suggesting keeping *splice* as the user-visible API,
but maybe plain sendfile could do this, and maybe someone would add
io_uring support, but all the complexity would be confined to the code
that does the actual copy and not spread to anywhere else in the
network stack.
--Andy
>
> sendfile() was a mistake. It is literally the "file->socket" thing
> that has been buggy.
>
> I absolutely refuse to get rid of splice code but keep the buggy sh*t
> cases that caused all the problems in the first place.
>
> Because *THAT* would just be completely insane and pointless.
>
> > Even if we’re just doing that, I continue to find it strange that we
> > require that a pipe be involved. What’s so special about pipes
>
> Again: it was never splice or the pipe that was the problem. Stop
> barking up the wrong tree.
>
> It was "file data to socket" that was the truly horrendous issue.
>
> That said, to explain the pipe: The reason for the pipe is to act as
> the kernel-side buffer.
>
> Now, these days we have much more capable iov_iter interfaces than we
> used to, and in that sense the "pipe as a buffer" is certainly not the
> obvious choice now.
>
> But even then you need to have a *handle* to the buffers for the
> general case, and that's what the pipe fd ends up then still
> effectively being.
>
> It was also done to avoid the M:N translation problem, because people
> wanted to do zero-copy between other things than just "file ->
> socket".
>
> But again: we're ABNSOLUTELY NOT keeping that "file -> socket" thing
> and getting rid of splice. That's literally keeping the bath-water
> and throwing out the baby.
>
> Splice is the *good* part (well, relatively - splice is bad too).
>
> ile->socket needs to DIE IN A FIRE considering the security problems it has had.
>
> I hope Jakub is right that the problems have been all fixed, and this
> is all theoretical, but having seen just *how* many there were, I'm a
> bit sceptical.
>
> Because if people think splice is complicated, you haven't looked at
> the skb rules. They are completely arbitrary and complex and spread
> all over the tree.
>
> Linus
--
Andy Lutomirski
AMA Capital Management, LLC
^ permalink raw reply
* Re: [PATCH 0/3] vmsplice: make vmsplice a trivial wrapper for preadv2/pwritev2
From: Linus Torvalds @ 2026-06-03 21:36 UTC (permalink / raw)
To: Andy Lutomirski
Cc: Askar Safin, akpm, axboe, brauner, david, dhowells, hch, jack,
linux-api, linux-fsdevel, linux-kernel, linux-mm, miklos, netdev,
patches, pfalcato, viro, willy
In-Reply-To: <CALCETrW3XcNLuB1Y6PSkxQDSK2o+=EB2AAd25SjWQqcJemwnbw@mail.gmail.com>
On Wed, 3 Jun 2026 at 14:31, Andy Lutomirski <luto@amacapital.net> wrote:
>
> I think I buried the lede too much and you're arguing against what I
> was trying not to say.
>
> Maybe we should keep an API that does an optimized copy, from one fd
> to another, that can send from a file to the network with at most ONE
> cpu-side copy. Not aiming for zero like sendfile / splice. Aiming
> for one.
Oh, absolutely - that's what my completely untested test patch basically did.
The user space interface was still there.
And the networking side still continued to use the ->splice_write()
thing for writing to the socket.
It was just the filesystem side that basically now instead of exposing
the page cache directly (with filemap_splice_read) now only exposed a
*copy* of the page cache (with copy_splice_read).
Linus
^ permalink raw reply
* Re: [PATCH 0/3] vmsplice: make vmsplice a trivial wrapper for preadv2/pwritev2
From: Linus Torvalds @ 2026-06-03 21:38 UTC (permalink / raw)
To: Andy Lutomirski
Cc: Askar Safin, akpm, axboe, brauner, david, dhowells, hch, jack,
linux-api, linux-fsdevel, linux-kernel, linux-mm, miklos, netdev,
patches, pfalcato, viro, willy
In-Reply-To: <CAHk-=wgn3QTLj+F+XccE10dXY-UGWN8+fNLEvhsLw+tik9rOmg@mail.gmail.com>
On Wed, 3 Jun 2026 at 14:36, Linus Torvalds
<torvalds@linux-foundation.org> wrote:
>
> It was just the filesystem side that basically now instead of exposing
> the page cache directly (with filemap_splice_read) now only exposed a
> *copy* of the page cache (with copy_splice_read).
... and let me note that UNTESTED part again.
The patch looked "ObviouslyCorrect(tm)" to me, and I did actually
compile-test it too.
So it probably wasn't _complete_ crap.
But I never even booted it, and if I had, I wouldn't have had any
loads that uses splice (or sendfile) anyway.
So caveat emptor.
Linus
^ permalink raw reply
* Re: [PATCH 0/3] vmsplice: make vmsplice a trivial wrapper for preadv2/pwritev2
From: Andy Lutomirski @ 2026-06-03 22:23 UTC (permalink / raw)
To: Linus Torvalds
Cc: Askar Safin, akpm, axboe, brauner, david, dhowells, hch, jack,
linux-api, linux-fsdevel, linux-kernel, linux-mm, miklos, netdev,
patches, pfalcato, viro, willy
In-Reply-To: <CAHk-=wgn3QTLj+F+XccE10dXY-UGWN8+fNLEvhsLw+tik9rOmg@mail.gmail.com>
On Wed, Jun 3, 2026 at 2:39 PM Linus Torvalds
<torvalds@linux-foundation.org> wrote:
>
> On Wed, 3 Jun 2026 at 14:31, Andy Lutomirski <luto@amacapital.net> wrote:
> >
> > I think I buried the lede too much and you're arguing against what I
> > was trying not to say.
> >
> > Maybe we should keep an API that does an optimized copy, from one fd
> > to another, that can send from a file to the network with at most ONE
> > cpu-side copy. Not aiming for zero like sendfile / splice. Aiming
> > for one.
>
> Oh, absolutely - that's what my completely untested test patch basically did.
>
> The user space interface was still there.
>
> And the networking side still continued to use the ->splice_write()
> thing for writing to the socket.
So I'm suspicious that you've possibly make bugs much (MUCH) harder to
exploit, but the underlying awful code and opportunity for bugs is
still there. MSG_SPLICE_PAGES is still around, and there is still
(AFAICS) no actual coherent description of what it means. There is
code that checks for it and apparently needs to do something special.
Foir example, some random kernel version I have checked out has this
delight in af_alg.c:
/* use the existing memory in an allocated page */
if (ctx->merge && !(msg->msg_flags & MSG_SPLICE_PAGES)) {
Grepping for MSG_SPLICE_PAGES come up with all kinds of terrors.
Check out the lovely comment in drivers/block/drbd/drbd_main.c, for
example...
And even with your patch, I think checking for MSG_SPLICE_PAGES still
matters: if I write to a pipe (using copy_splice_read or even just
plain write) and then I tee() that data, then I splice one of those
teed copies into a socket, then we hit ->sendmsg with MSG_SPLICE_PAGES
set, and we're hoping that the code does the right thing. And maybe
all the bugs are fixed by now or maybe they're not. Most of what your
patch accomplishes is breaking the connection between the buffers and
pagecache, so you can't poison /sbin/su.
It also seems kind of unfortunate that we can have skbs that contain
data that isn't actually owned by the socket in question, and, with
your patch applied, I'm wondering if the only case where this can
really happen is tee() and a handful of random drivers that send to
sockets. (The ones in drivers/nvme/host/tcp.c and iSCSI seem like the
ones that people are likely to care about the most.)
I *think* that what I'm sort of suggesting is to drop this ability
from the kernel as well, or at least to consider it. skbs would
always own their contents. And something would get wired up so that
at least the cases of sendfile, nvme and iscsi to TCP or UDP sockets
would still works with only one copy, from the source page cache into
the socket buffer.
I suppose the counterargument is that, even if more bugs exist, it's a
bit hard to imagine a real attack involving tee, and one needs
privileges to set up nvme or iscsi aimed at an unusual socket type.
--
Andy Lutomirski
AMA Capital Management, LLC
^ permalink raw reply
* Re: [PATCH 0/3] vmsplice: make vmsplice a trivial wrapper for preadv2/pwritev2
From: Askar Safin @ 2026-06-03 22:43 UTC (permalink / raw)
To: luto
Cc: akpm, axboe, brauner, david, dhowells, hch, jack, linux-api,
linux-fsdevel, linux-kernel, linux-mm, miklos, netdev, patches,
pfalcato, safinaskar, torvalds, viro, willy
In-Reply-To: <CALCETrW3XcNLuB1Y6PSkxQDSK2o+=EB2AAd25SjWQqcJemwnbw@mail.gmail.com>
Andy Lutomirski <luto@amacapital.net>:
> Maybe we should keep an API that does an optimized copy, from one fd
> to another, that can send from a file to the network with at most ONE
> cpu-side copy. Not aiming for zero like sendfile / splice. Aiming
> for one.
Yes, this is what my hypothetical future patch will do.
One copy from pagecache to pipe, and then network uses that buffer
directly.
> But splice_to_socket involves
> MSG_SPLACE_PAGES, which I think is a part of the mess that you
> dislike. And the path where one does copy_splice_read and then
> splice_to_socket has to be a bit complex because of tee and (I think)
> because splice_to_socket cannot assume that the incoming data is just
> ordinary unshared buffers.
My future patch will provide new guarantee: pipe buffers are always
stable, i. e. they will not be externally-modified.
So hopefully network code will be adjusted to use this guarantee.
But pipe buffers will not be "ordinary unshared buffers".
They still may be shared with other things because of tee(2).
(But they are still stable! They will not be randomly modified!)
But network code can do "pipe_buf_try_steal" and thus ensure that
these buffers are not shared with anything else.
So, network code can be modified to use "pipe_buf_try_steal", and you
will get "ordinary unshared buffers" exactly as you want. This will
give you in total exactly one copy.
Also: as well as I understand, previously, pipe_buf_try_steal was
kind of lie. It may return true for buffers created via vmsplice with
GIFT. (I did not check this, but I think so.) I. e. pipe_buf_try_steal will
return "true" in this case, but pages are still shared! But, thanks to my
vmsplice patchset (which is already applied), this is no longer true!
So now pipe_buf_try_steal is absolutely safe to use!
Finally, we can degrade tee(2) to copy, and hopefully this will
allow us to always be sure that pipe buffers are not shared with anything.
This is possible future direction.
--
Askar Safin
^ permalink raw reply
* Re: [PATCH 0/3] vmsplice: make vmsplice a trivial wrapper for preadv2/pwritev2
From: Andy Lutomirski @ 2026-06-03 22:49 UTC (permalink / raw)
To: Askar Safin
Cc: akpm, axboe, brauner, david, dhowells, hch, jack, linux-api,
linux-fsdevel, linux-kernel, linux-mm, miklos, netdev, patches,
pfalcato, torvalds, viro, willy
In-Reply-To: <20260603224311.834796-1-safinaskar@gmail.com>
On Wed, Jun 3, 2026 at 3:43 PM Askar Safin <safinaskar@gmail.com> wrote:
>
> Andy Lutomirski <luto@amacapital.net>:
> > Maybe we should keep an API that does an optimized copy, from one fd
> > to another, that can send from a file to the network with at most ONE
> > cpu-side copy. Not aiming for zero like sendfile / splice. Aiming
> > for one.
>
> Yes, this is what my hypothetical future patch will do.
>
> One copy from pagecache to pipe, and then network uses that buffer
> directly.
>
> > But splice_to_socket involves
> > MSG_SPLACE_PAGES, which I think is a part of the mess that you
> > dislike. And the path where one does copy_splice_read and then
> > splice_to_socket has to be a bit complex because of tee and (I think)
> > because splice_to_socket cannot assume that the incoming data is just
> > ordinary unshared buffers.
>
> My future patch will provide new guarantee: pipe buffers are always
> stable, i. e. they will not be externally-modified.
>
> So hopefully network code will be adjusted to use this guarantee.
>
> But pipe buffers will not be "ordinary unshared buffers".
>
> They still may be shared with other things because of tee(2).
> (But they are still stable! They will not be randomly modified!)
>
> But network code can do "pipe_buf_try_steal" and thus ensure that
> these buffers are not shared with anything else.
>
> So, network code can be modified to use "pipe_buf_try_steal", and you
> will get "ordinary unshared buffers" exactly as you want. This will
> give you in total exactly one copy.
>
> Also: as well as I understand, previously, pipe_buf_try_steal was
> kind of lie. It may return true for buffers created via vmsplice with
> GIFT. (I did not check this, but I think so.) I. e. pipe_buf_try_steal will
> return "true" in this case, but pages are still shared! But, thanks to my
> vmsplice patchset (which is already applied), this is no longer true!
> So now pipe_buf_try_steal is absolutely safe to use!
>
> Finally, we can degrade tee(2) to copy, and hopefully this will
> allow us to always be sure that pipe buffers are not shared with anything.
> This is possible future direction.
I'm a bit nervous that, if I've read the code correctly (a big if),
then iscsi and nvme will still send *shared* buffers via
MSG_SPLICE_PAGES, but that normal user code will not be able to do
this, and that something will bitrot.
^ permalink raw reply
* Re: [PATCH 0/3] vmsplice: make vmsplice a trivial wrapper for preadv2/pwritev2
From: Linus Torvalds @ 2026-06-03 22:53 UTC (permalink / raw)
To: Andy Lutomirski
Cc: Askar Safin, akpm, axboe, brauner, david, dhowells, hch, jack,
linux-api, linux-fsdevel, linux-kernel, linux-mm, miklos, netdev,
patches, pfalcato, viro, willy
In-Reply-To: <CALCETrXpqPMS487Bm8f8mHe8hv9DzCqoaW4UdoHetzYBUAhYLw@mail.gmail.com>
On Wed, 3 Jun 2026 at 15:23, Andy Lutomirski <luto@amacapital.net> wrote:
>
> So I'm suspicious that you've possibly make bugs much (MUCH) harder to
> exploit, but the underlying awful code and opportunity for bugs is
> still there. MSG_SPLICE_PAGES is still around, and there is still
> (AFAICS) no actual coherent description of what it means.
I don't disagree. I've only looked at the filesystem side.
The networking side does some odd stuff too (and I did look at some of
that, and had to be edumacated by Jakub on some of the subtler rules
for what skb data sharing is ok and when it's not - really not my
area).
But at least MSG_SPLICE_PAGES should be kernel-internal only
interface, and once you don't share page cache pages with networking
code I think that kneecaps a lot of the attacks.
So that's really the aim here for me - at least _attempting_ to go
"maybe we can just limit splice enough that it doesn't even *matter*
when networking does something odd and questionable".
And it's entirely possible that the current zero-copy "networking gets
direct access to the page cache folios" is a huge and insurmountable
performance requirement for some loads. So the vmsplice patch - and
_particularly_ my suggested "let's try always copying" patch - may
simply be doomed.
But I'd rather try to simplify the splice code by removing complexity
- and possibly then failing and having to revert it and rethink things
- than not even trying.
Because I think splice() is a *cool* feature. It was always *clever*.
I just don't think it's worth the pain it has cause.
And it's been around for a long long time, and after more than two
decades it's still most definitely not _widely_ used.
So that makes it a failure in my book. Sometimes "clever" just isn't
the right thing.
Linus
^ permalink raw reply
* Re: [RFC PATCH v5 1/2] vfs: add O_CREAT|O_DIRECTORY to open*(2)
From: NeilBrown @ 2026-06-03 22:56 UTC (permalink / raw)
To: Jori Koolstra, linux-api
Cc: Christian Brauner, Aleksa Sarai, Alexander Viro, Jan Kara,
linux-kernel, linux-fsdevel, cmirabil
In-Reply-To: <1571071834.388026.1780492561946@kpc.webmail.kpnmail.nl>
On Wed, 03 Jun 2026, Jori Koolstra wrote:
> > Op 02-06-2026 17:44 CEST schreef Christian Brauner <brauner@kernel.org>:
> >
> > Yes, I agree. This would change error codes but I don't think it
> > matters:
> >
> > * O_WRONLY | O_DIRECTORY on non-directory -> ENOTDIR
> > * O_WRONLY | O_DIRECTORY on directory -> EISDIR
> >
> > I don't think that really matters and we should be able to collapse this
> > to ENOTDIR.
>
> I will pick this up in the next version of O_CREAT|O_DIRECTORY. I think that
> makes most sense.
>
> I have an outstanding patch for changing the EACCES/EPERM to EOPNOTSUPP;
> Jeff and Jan were skeptical, but I want to know your opinion as well.
> I feel the the scenario where userspace has no fall-through but does
> handle every single -E listed in the man-page quite unlikely, so I say
> lets change them and we'll hear from them if somehow someone relied on
> this weird way of error handling.
Please cc linux-api@vger.kernel.org on code and discussions that involve
API changes. I have cc:ed them on this reply.
Thanks,
NeilBrown
>
> > > >
> > > > There is another point, I maybe should have mentioned in the cover letter: I have not attempted
> > > > to handle dangling symlinks for O_MKDIR. Not because I think they are a great idea (as Aleksa
> > > > has mentioned, but I am not very familiar with the dragons it entails), but I wanted to discuss
> > > > what behavior we want in this case. Do we say that we never do a mkdir after following a lookup
> > > > last symlink? I don't think that state is even recorded right now.
> > >
> > > I think the state might be recorded in nd->depth. But you probably
> > > don't want to use that directly. Maybe forcing LOOKUP_FOLLOW to be
> > > cleared if O_CREAT|O_DIRECTORY is set would be good. But what would
> > > stop you opening an existing directory through a symlink....
> > >
> > > Probably we need a clear statement of intended semantics which we can
> > > review, agree on, then implement. Have you looked at preparing a patch
> > > for man-pages to document the change in behaviour for openat etc?
> >
> > Ugh, dangling symlinks. Actually, scratch that: Ugh, symlinks. So
> > O_CREAT without O_NOFOLLOW allows you to create the target of a dangling
> > symlink iirc. I always forget that. I think this is a very subtle bug
> > and maybe - with both eyes closed - a feature at times.
> >
> > We should straighten the behavior for O_DIRECTORY | O_CREAT and we
> > agreed on that during LSFMM. It would be nice if we could get away with
> > simply implying O_NOFOLLOW but I think you're right, Neil, that this
> > prevents a valid O_CREAT | O_DIRECTORY on an existing directory which we
> > can't do. Makes this kind of a pointless excercise.
> >
> > But this shouldn't be all that crazy to do right. Using the O_CREAT as
> > an _example_ for what we'd need:
> >
> > fs: refuse O_CREAT through a dangling symlink
> >
> > open(O_CREAT) without O_EXCL follows a trailing symlink and, when the
> > symlink target does not exist, creates it. Refuse to create through a
> > dangling symlink instead.
> >
> > In lookup_open() a negative target reached with nd->depth > 0 was
> > arrived at by following a trailing symlink; since the dentry is negative
> > the symlink is dangling. Set create_error to -ELOOP in that case.
> > Reusing the existing create_error path strips O_CREAT for both the
> > generic and ->atomic_open create paths and only reports the error when
> > the target is actually negative, so opening an existing target through a
> > symlink, interior symlinks, and O_EXCL (which never follows the trailing
> > link) are all unaffected.
> >
> > Hastily-Cobbled-Together-by: Christian Brauner (Amutable) <brauner@kernel.org>
> >
> > diff --git a/fs/namei.c b/fs/namei.c
> > index c7fac83c9a85..d20bbcc7e8d3 100644
> > --- a/fs/namei.c
> > +++ b/fs/namei.c
> > @@ -4468,6 +4468,9 @@ static struct dentry *lookup_open(struct nameidata *nd, struct file *file,
> > dentry, mode);
> > else
> > create_error = -EROFS;
> > + /* refuse to create through a dangling (trailing) symlink */
> > + if (unlikely(nd->depth) && !create_error)
> > + create_error = -ELOOP;
> > }
> > if (create_error)
> > open_flag &= ~O_CREAT;
> >
> > It can't be that easy...
>
> This is what I suggested above, correct, in terms of behavior?
>
> In terms of the patch, I think this will work, but struct nameidata could really
> use some commentary for its fields. I spent the last two hours verifying that
> nd->depth really does what I thought it did, and I am still not 100% positive.
> AFAIS, nd->depth indeed tracks the current symlink depth, which outside of
> link_path_walk() reduces to the number of trailing links followed.
>
> But if Neil's rework of lookup_open() is merged we lose access here to nd.
> @Neil, have you thought about what would be a good way to resolve that?
>
^ permalink raw reply
* Re: [PATCH 0/3] vmsplice: make vmsplice a trivial wrapper for preadv2/pwritev2
From: Askar Safin @ 2026-06-03 23:00 UTC (permalink / raw)
To: luto
Cc: akpm, axboe, brauner, david, dhowells, hch, jack, linux-api,
linux-fsdevel, linux-kernel, linux-mm, miklos, netdev, patches,
pfalcato, safinaskar, torvalds, viro, willy
In-Reply-To: <CALCETrU94ja56CA5CRtXrm1v_7gBaPUNOHKQzS=JNF9JZ7Fznw@mail.gmail.com>
Andy Lutomirski <luto@amacapital.net>:
> On Wed, Jun 3, 2026 at 3:43 PM Askar Safin <safinaskar@gmail.com> wrote:
> > Finally, we can degrade tee(2) to copy, and hopefully this will
> > allow us to always be sure that pipe buffers are not shared with anything.
> > This is possible future direction.
>
> I'm a bit nervous that, if I've read the code correctly (a big if),
> then iscsi and nvme will still send *shared* buffers via
> MSG_SPLICE_PAGES, but that normal user code will not be able to do
> this, and that something will bitrot.
As well as I understand you correctly, you mean that if we remove
tee(2), then there still will be subsystems, which will be able to
send shared pages.
Yes, I totally agree.
So, if we remove tee(2), then we will probably need to remove all
non-standard implementations of pipe_buf_operations.
--
Askar Safin
^ permalink raw reply
* Re: [PATCH 0/3] vmsplice: make vmsplice a trivial wrapper for preadv2/pwritev2
From: Linus Torvalds @ 2026-06-04 0:01 UTC (permalink / raw)
To: Askar Safin
Cc: luto, akpm, axboe, brauner, david, dhowells, hch, jack, linux-api,
linux-fsdevel, linux-kernel, linux-mm, miklos, netdev, patches,
pfalcato, viro, willy
In-Reply-To: <20260603230122.851517-1-safinaskar@gmail.com>
On Wed, 3 Jun 2026 at 16:01, Askar Safin <safinaskar@gmail.com> wrote:
>
> So, if we remove tee(2), then we will probably need to remove all
> non-standard implementations of pipe_buf_operations.
I don't think tee matters.
Sure, it will share pages across pipes.
But if we make normal "splice to pipe" always copy from the page
cache, nobody cares.
You can corrupt the resulting pages as much as you want - through
multiple pipes if you use tee() to copy it - and it's all just
corrupting your private copy.
And yes, iSCSI and nvme might do their own splice-like thing, but
again, nobody really cares. When it's all kernel-internal, the attack
surface has gone away.
So that's why splice() (and vmsplice()) is special - not because it's
buggy, but because it's the user-facing attack surface to expose bugs
elsewhere.
Linus
^ permalink raw reply
* Re: [PATCH 0/3] vmsplice: make vmsplice a trivial wrapper for preadv2/pwritev2
From: Askar Safin @ 2026-06-04 0:45 UTC (permalink / raw)
To: safinaskar
Cc: akpm, axboe, brauner, david, dhowells, hch, jack, linux-api,
linux-fsdevel, linux-kernel, linux-mm, miklos, netdev, patches,
pfalcato, torvalds, viro, willy
In-Reply-To: <20260531010107.1953702-1-safinaskar@gmail.com>
Askar Safin <safinaskar@gmail.com>:
> For all these reasons I propose to make vmsplice a simple wrapper for
> preadv2/pwritev2.
This patchset is already in next, but I still kindly ask people to
carefully review it. I'm still a new contributor, and I can make mistakes.
For example, in vmsplice I do "CLASS(fd, f)(fd)" and then I pass
"fd" (i. e. integer) to "do_writev/do_readv". I don't know whether
this is okay to do so.
--
Askar Safin
^ permalink raw reply
* Re: [PATCH 0/3] vmsplice: make vmsplice a trivial wrapper for preadv2/pwritev2
From: Linus Torvalds @ 2026-06-04 1:52 UTC (permalink / raw)
To: Askar Safin
Cc: akpm, axboe, brauner, david, dhowells, hch, jack, linux-api,
linux-fsdevel, linux-kernel, linux-mm, miklos, netdev, patches,
pfalcato, viro, willy
In-Reply-To: <20260604004559.1112474-1-safinaskar@gmail.com>
On Wed, 3 Jun 2026 at 17:46, Askar Safin <safinaskar@gmail.com> wrote:
>
> For example, in vmsplice I do "CLASS(fd, f)(fd)" and then I pass
> "fd" (i. e. integer) to "do_writev/do_readv". I don't know whether
> this is okay to do so.
Oh, good point.
It's ok in the sense that it will work, and it's not really going to
cause problems, but it does mean that the 'struct file' will be looked
up twice.
And *technically* it's a TOCTOU race, where the first time you look it
up - in the vmsplice() wrapper - it could be one file, and you make
decisions based on that. And then pass it off to do_writev(), and it
will look it up again, and now it might be a different file.
Does it *matter*? No. Even if the file changed, and is now something
else, it's just going to be a different file that the user does
writev() on. do_writev() will still do all the appropriate safety
checks etc, so it doesn't really change anything. It just means that
you could pass what you *think* is a pipe (because you did that
+ if (!get_pipe_info(fd_file(f), /* for_splice = */ false))
+ return -EBADF;
and by the time do_writev() then looks up the fd again it might be
something else, and now the user used vmsplice() as a really odd way
to write to a another non-pipe file instead. But the user could have
done that with a regular writev(), so it's just the user being silly -
not something that really confuses the kernel.
Coimpletely harmless, in other words.
But it would probably be *cleaner* to pass in the 'struct file *'
pointer that you already looked up once instead, and use vfs_writev()
instead of do_writev().
And I do suspect that the wrapper system call should use the same
SYSCALL_DEFINE4(vmsplice, int, fd, ..
that the original used. Because it somebody crazy had the high bits
set in 'fd', the old vmsplice() system call didn't care, but your new
emulation system call will actually see the high bits on a 64-bit
architecture.
Again - that doesn't actually *matter*, because "CLASS(fd)" takes an
"int fd" and those high bits will be masked out at use time both in
vmsplice() and in do_readv/writev().
So it won't affect any behavior, but it does look a bit odd in the conversion.
And I already answered Christian wrt the change in behavior: I think
RWF_NOWAIT should always be set on the writing side - because splice()
never waited after it filled a pipe - and instead that
SPLICE_F_NONBLOCK flag should be used before write to check for
whether we'll wait *before* doing the write like it used to do with
ret = wait_for_space(pipe, flags);
in vmsplice_to_pipe().
(On the other side, vmsplice_from_pipe() used to do
pipe_clear_nowait(), but I think that becomes a non-issue with the
conversion to readv()).
And once you need wait_for_space(), that probably means that the new
vmsplice() wrapper simpler needs to remain inside fs/splice.c, and we
just need to make vfs_readv/vfs_writev non-static.
Linus
^ permalink raw reply
* Re: [PATCH 0/3] vmsplice: make vmsplice a trivial wrapper for preadv2/pwritev2
From: Willy Tarreau @ 2026-06-04 6:32 UTC (permalink / raw)
To: Andrew Morton
Cc: Steven Rostedt, Al Viro, Linus Torvalds, Christian Brauner,
Askar Safin, linux-kernel, linux-mm, linux-api, netdev,
Matthew Wilcox, Jens Axboe, Christoph Hellwig, David Howells,
David Hildenbrand, Pedro Falcato, Miklos Szeredi, patches,
linux-fsdevel, Jan Kara
In-Reply-To: <20260601172825.a51a588ec1c32617a0e12d78@linux-foundation.org>
On Mon, Jun 01, 2026 at 05:28:25PM -0700, Andrew Morton wrote:
> On Mon, 1 Jun 2026 16:04:55 -0400 Steven Rostedt <rostedt@goodmis.org> wrote:
>
> > On Mon, 1 Jun 2026 18:33:25 +0100
> > Al Viro <viro@zeniv.linux.org.uk> wrote:
> >
> > > On Mon, Jun 01, 2026 at 10:17:23AM -0700, Linus Torvalds wrote:
> > >
> > > > TLDR: maybe we could ghet rid of "f_op->splice_read". *That* would be
> > > > a big simplification.
> > >
> > > FUSE might be interesting - fuse_dev_splice_read() and its ilk.
> > > Communications between the kernel and fuse server at least used to
> > > seriously want that, so that would be one place to look for unhappy
> > > userland...
> > >
> > > splice-related logics in fs/fuse/dev.c is interesting; another place
> > > like this is kernel/trace/, but I'm less familiar with that one.
> > >
> > > rostedt Cc'd (miklos already had been)
> >
> > Thanks for the Cc. The tracing ring buffer was specifically made to be used
> > by splice and the libtracefs has a lot of code to use it as well. As
> > reading the ring buffer literally swaps out the write portion with a blank
> > read portion, that portion (sub-buffer) is used to be directly fed into
> > splice, providing a zero-copy of the trace data from the write of the event
> > to going into a file.
> >
> > trace-cmd defaults to using splice to copy the tracing ring buffer directly
> > into files to avoid as much copying during live recordings as possible.
> >
> > Whatever changes we make, I would like to make sure there's no regressions
> > in performance of trace-cmd record.
>
> Well yes, The patchset seems sensible from a quality POV. But to make
> a decision we should first have a decent understanding of its downside
> impact.
>
> I haven't seen a description of that impact in the discussion thus far.
> And that description is owed, please.
>
> I assume a small number of specialized applications are using
> vmsplice() to great effect? What are those applications? What is the
> impact of this change?
> Once we are armed with that information, is there some middle ground in
> which we de-feature vmsplice()? Fall back to pread/pwrite in the
> tricky cases and still permit vmsplicing if the application is
> appropriately restrictive in it usage?
I'm using vmsplice() + tee() + splice() in high-performance applications,
load generators to be precise, and soon a cache. This is super convenient
and extremely efficient:
- vmsplice() is used to prepare a "master" pipe with data to be sent
over TCP or kTLS
- then for each request, we do tee() from this master pipe to per-request
pipes.
- the per-request pipes are those that are used to deliver the data to
the socket via splice().
So we effectively use vmsplice(), tee() and splice() here, and for exactly
the reasons they were designed: only play with page refcount and not copy
data. The code is here for the curious:
https://git.haproxy.org/?p=haproxy.git;a=blob;f=src/haterm.c
and its ancestor is here:
https://github.com/wtarreau/httpterm/blob/master/httpterm.c
It simply doubles the network bandwidth compared to not using that.
(62 Gbps per core vs 31). I would seriously miss it if I couldn't use
this anymore.
I also have mid-term plans for using vmsplice() to deliver contents from
a cache to sockets as well via splice(). Right now our cache is split into
too small chunks (1kB) to make that useful, but as soon as we can move to
4kB pages, it will make sense. There the same gains are expected, and I
would particularly dislike the idea of no longer being able to implement
zero-copy!
Maybe some arrangements are possible though. I'm not seeing any other way
to achieve the same things differently, but possibly that the base of the
problem is the easy abuse of vmsplice() to affect the page cache. Maybe
placing certain restrictions such as he area only being mapped to anonymous
pages, or anything similar could make sense. In my use case it wouldn't be
that much of a constraint. Well, for the cache maybe it could be though,
as it would prevent us from sharing it via persistent storage. Or maybe
we could require a CAP_BACKED_VMSPLICE to be allowed to vmsplice file-
backed pages, which could be sufficient to prevent easy LPE each time a
bug is found ?
I think that the users of this APIs are rare enough that we can probably
find a solution that anyone can reasonably adapt to with minimal
constraints. But most likely each of these few users rely on this
*a lot*.
Just my two cents,
Willy
^ permalink raw reply
* Re: [PATCH 2/3] vmsplice: make vmsplice a trivial wrapper for preadv2/pwritev2
From: David Laight @ 2026-06-04 9:06 UTC (permalink / raw)
To: Askar Safin
Cc: metze, akpm, axboe, brauner, david, dhowells, hch, jack,
linux-api, linux-fsdevel, linux-kernel, linux-mm, miklos, netdev,
patches, pfalcato, torvalds, viro, willy
In-Reply-To: <20260603211736.755139-1-safinaskar@gmail.com>
On Thu, 4 Jun 2026 00:17:36 +0300
Askar Safin <safinaskar@gmail.com> wrote:
> Stefan Metzmacher <metze@samba.org>:
> > Why is 'int fd' changed to 'unsigned long fd'?
>
> Because preadv2 and pwritev2 take "unsigned long". I want vmsplice
> to be as similar as possible to preadv2 and pwritev2.
Something needs to ensure that the high 32bits of the fd get masked off
on 64bit systems.
They can be non-zero in the register that comes from userspace.
-- David
>
> > Should that be its own commit if the change is desired?
>
> Yes, possibly. But this patchset already got to next.
>
^ permalink raw reply
* Re: [PATCH 2/3] vmsplice: make vmsplice a trivial wrapper for preadv2/pwritev2
From: Linus Torvalds @ 2026-06-04 14:17 UTC (permalink / raw)
To: David Laight
Cc: Askar Safin, metze, akpm, axboe, brauner, david, dhowells, hch,
jack, linux-api, linux-fsdevel, linux-kernel, linux-mm, miklos,
netdev, patches, pfalcato, viro, willy
In-Reply-To: <20260604100609.6b37f500@pumpkin>
On Thu, 4 Jun 2026 at 02:06, David Laight <david.laight.linux@gmail.com> wrote:
>
> Something needs to ensure that the high 32bits of the fd get masked off
> on 64bit systems.
That something already exists: CLASS(fd, f)(fd);
It ignores the top bits, because 'fdget()' takes an 'unsigned int'.
We have been a bit random in how we declare the system calls in
general, and we mix 'unsigned int' and 'int' and 'unsigned long'
pretty much randomly when it comes to file descriptor arguments to
system calls.
fs/read_write.c in particular uses all three cases with no real logic to it all:
SYSCALL_DEFINE3(lseek, unsigned int, fd, ..
SYSCALL_DEFINE3(readv, unsigned long, fd, ..
SYSCALL_DEFINE4(sendfile, int, out_fd, ..
but then anything that uses fdget() (through one of the helper classes
or not) will simply not care.
Does it make sense? Is it pretty? Nope. Does it matter? Also nope.
Linus
^ permalink raw reply
* Re: [PATCH 0/3] vmsplice: make vmsplice a trivial wrapper for preadv2/pwritev2
From: Linus Torvalds @ 2026-06-04 14:31 UTC (permalink / raw)
To: Willy Tarreau
Cc: Andrew Morton, Steven Rostedt, Al Viro, Christian Brauner,
Askar Safin, linux-kernel, linux-mm, linux-api, netdev,
Matthew Wilcox, Jens Axboe, Christoph Hellwig, David Howells,
David Hildenbrand, Pedro Falcato, Miklos Szeredi, patches,
linux-fsdevel, Jan Kara
In-Reply-To: <aiEb8CTM-ovMIq7-@1wt.eu>
On Wed, 3 Jun 2026 at 23:32, Willy Tarreau <w@1wt.eu> wrote:
>
> I'm using vmsplice() + tee() + splice() in high-performance applications,
> load generators to be precise, and soon a cache. This is super convenient
> and extremely efficient:
>
> - vmsplice() is used to prepare a "master" pipe with data to be sent
> over TCP or kTLS
> - then for each request, we do tee() from this master pipe to per-request
> pipes.
> - the per-request pipes are those that are used to deliver the data to
> the socket via splice().
So most of those would actually not be affected by any of the existing
patches: the pipe->socket splice would remain, the tee() code would
still just take a ref to the page count.
The vmsplice() would change, but looking at your haterm.c sources, it
looks like it's mostly a fairly small thing ("common_response[]" being
16kB).
That is typically *faster* to just copy than look up pages.
HOWEVER.
It looks like you're actually doing exactly the thing that I thought
was crazy and wouldn't even work reliably: you change the
common_response[] contents dynamically *after* the vmsplice, and
depend on the fact that changing it in user space changes the buffer
in the pipe too.
So that would break *entirely* with the vmsplice() changes if I read
the code right (which I might not do) simply because that looks like
it really does require that "wrutably shared buffer after the fact".
Interesting. Because the vmsplice() code uses get_user_pages_fast(),
and honestly, it never pinned the page reliably to the original source
- it breaks COW randomly in one direction or the other after fork()
(and I thouht even after a page-out, but thinking more about it the
swap cache may have made it work for that case).
Uhhuh. That does look like it makes the vmsplice() changes untenable.
But I may be reading your haproxy code entirely wrong.
Linus
^ permalink raw reply
* Re: [PATCH 0/3] vmsplice: make vmsplice a trivial wrapper for preadv2/pwritev2
From: Andy Lutomirski @ 2026-06-04 15:53 UTC (permalink / raw)
To: Willy Tarreau
Cc: Andrew Morton, Steven Rostedt, Al Viro, Linus Torvalds,
Christian Brauner, Askar Safin, linux-kernel, linux-mm, linux-api,
netdev, Matthew Wilcox, Jens Axboe, Christoph Hellwig,
David Howells, David Hildenbrand, Pedro Falcato, Miklos Szeredi,
patches, linux-fsdevel, Jan Kara
In-Reply-To: <aiEb8CTM-ovMIq7-@1wt.eu>
On Wed, Jun 3, 2026 at 11:32 PM Willy Tarreau <w@1wt.eu> wrote:
>
> On Mon, Jun 01, 2026 at 05:28:25PM -0700, Andrew Morton wrote:
> > On Mon, 1 Jun 2026 16:04:55 -0400 Steven Rostedt <rostedt@goodmis.org> wrote:
> >
> > > On Mon, 1 Jun 2026 18:33:25 +0100
> > > Al Viro <viro@zeniv.linux.org.uk> wrote:
> > >
> > > > On Mon, Jun 01, 2026 at 10:17:23AM -0700, Linus Torvalds wrote:
> > > >
> > > > > TLDR: maybe we could ghet rid of "f_op->splice_read". *That* would be
> > > > > a big simplification.
> > > >
> > > > FUSE might be interesting - fuse_dev_splice_read() and its ilk.
> > > > Communications between the kernel and fuse server at least used to
> > > > seriously want that, so that would be one place to look for unhappy
> > > > userland...
> > > >
> > > > splice-related logics in fs/fuse/dev.c is interesting; another place
> > > > like this is kernel/trace/, but I'm less familiar with that one.
> > > >
> > > > rostedt Cc'd (miklos already had been)
> > >
> > > Thanks for the Cc. The tracing ring buffer was specifically made to be used
> > > by splice and the libtracefs has a lot of code to use it as well. As
> > > reading the ring buffer literally swaps out the write portion with a blank
> > > read portion, that portion (sub-buffer) is used to be directly fed into
> > > splice, providing a zero-copy of the trace data from the write of the event
> > > to going into a file.
> > >
> > > trace-cmd defaults to using splice to copy the tracing ring buffer directly
> > > into files to avoid as much copying during live recordings as possible.
> > >
> > > Whatever changes we make, I would like to make sure there's no regressions
> > > in performance of trace-cmd record.
> >
> > Well yes, The patchset seems sensible from a quality POV. But to make
> > a decision we should first have a decent understanding of its downside
> > impact.
> >
> > I haven't seen a description of that impact in the discussion thus far.
> > And that description is owed, please.
> >
> > I assume a small number of specialized applications are using
> > vmsplice() to great effect? What are those applications? What is the
> > impact of this change?
>
> > Once we are armed with that information, is there some middle ground in
> > which we de-feature vmsplice()? Fall back to pread/pwrite in the
> > tricky cases and still permit vmsplicing if the application is
> > appropriately restrictive in it usage?
>
> I'm using vmsplice() + tee() + splice() in high-performance applications,
> load generators to be precise, and soon a cache. This is super convenient
> and extremely efficient:
>
> - vmsplice() is used to prepare a "master" pipe with data to be sent
> over TCP or kTLS
> - then for each request, we do tee() from this master pipe to per-request
> pipes.
> - the per-request pipes are those that are used to deliver the data to
> the socket via splice().
>
> So we effectively use vmsplice(), tee() and splice() here, and for exactly
> the reasons they were designed: only play with page refcount and not copy
> data. The code is here for the curious:
>
> https://git.haproxy.org/?p=haproxy.git;a=blob;f=src/haterm.c
>
> and its ancestor is here:
>
> https://github.com/wtarreau/httpterm/blob/master/httpterm.c
>
> It simply doubles the network bandwidth compared to not using that.
> (62 Gbps per core vs 31). I would seriously miss it if I couldn't use
> this anymore.
>
Wait a moment. This is neat, but it's literally just a benchmark,
right? I skimmed the code, and it doesn't look like a production
workload, either. And you manage to get around the awfulness of the
vmsplice API's complete failure to tell you when it's done with a
buffer by ... never actually changing the contents of the buffer. Do
you have any idea how you would write correct code that uses vmsplice
for sends and then *ever* mutates the data without literally
munmapping (or madvise or something) the data do you can safely mutate
it?
> I also have mid-term plans for using vmsplice() to deliver contents from
> a cache to sockets as well via splice(). Right now our cache is split into
> too small chunks (1kB) to make that useful, but as soon as we can move to
> 4kB pages, it will make sense. There the same gains are expected, and I
> would particularly dislike the idea of no longer being able to implement
> zero-copy!
If I'm understanding you correctly, you see (and measured!) a
performance improvement, and you would like to use it in production.
It seems to me that this is an excellent opportunity to remember that
vmsplice gets a performance boost in a highly synthetic situation that
sort of resembles a cache scenario and then to deprecate vmsplice and
build something better! Or discover that we already have something
better, perhaps :)
https://man7.org/linux/man-pages/man3/io_uring_prep_send_zc.3.html
I see that this can submit a buffer without a syscall (tee + splice is
*two* syscalls!) and that it has directly addressed what I see as the
really big deficiency in vmsplice: "This second notification tells the
application that the memory associated with the send is safe to get
reused." If I were writing the user code, I would very much want that
notification to be an explicit part of the API instead of making a
wild guess as I think I would need to do with vmsplice.
--Andy
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox