Fwd: Fwd: [RFC PATCH 1/4] splice: Fix corruption of spliced data after splice() returns

Fwd: Fwd: [RFC PATCH 1/4] splice: Fix corruption of spliced data after splice() returns

[Alejandro Colomar]

[-- Attachment #1.1: Type: text/plain, Size: 2641 bytes --]

Hi Randy!

Thanks!  I'll FWD to the linux-man@ mailing list too.

Cheers,
Alex


-------- Forwarded Message --------
Subject: Fwd: [RFC PATCH 1/4] splice: Fix corruption of spliced data after splice() returns
Date: Wed, 19 Jul 2023 17:36:03 -0700
From: Randy Dunlap <rdunlap@infradead.org>
To: Alejandro Colomar <alx.manpages@gmail.com>, Michael Kerrisk <mtk.manpages@gmail.com>

FYI:


-------- Forwarded Message --------
Subject: Re: [RFC PATCH 1/4] splice: Fix corruption of spliced data after splice() returns
Date: Wed, 19 Jul 2023 17:00:17 -0700
From: Linus Torvalds <torvalds@linux-foundation.org>
To: Matt Whitlock <kernel@mattwhitlock.name>
CC: Matthew Wilcox <willy@infradead.org>, Miklos Szeredi <miklos@szeredi.hu>, David Howells <dhowells@redhat.com>, netdev@vger.kernel.org, Dave Chinner <david@fromorbit.com>, Jens Axboe <axboe@kernel.dk>, linux-fsdevel@kvack.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Christoph Hellwig <hch@lst.de>, linux-fsdevel@vger.kernel.org

On Wed, 19 Jul 2023 at 16:41, Matt Whitlock <kernel@mattwhitlock.name> wrote:
>
> Then that is my request. This entire complaint/discussion/argument would
> have been avoided if splice(2) had contained a sentence like this one from
> sendfile(2):
>
> "If out_fd refers to a socket or pipe with zero-copy support, callers must
> ensure the transferred portions of the file referred to by in_fd remain
> unmodified until the reader on the other end of out_fd has consumed the
> transferred data."
>
> That is a clear warning of the perils of the implementation under the hood,
> and it could/should be copied, more or less verbatim, to splice(2).

Ack. Internally in the kernel, the two really have always been more or
less of intermingled.

In fact, I think splice()/sendfile()/tee() could - and maybe should -
actually be a single man-page to make it clear that they are all
facets of the same thing.

The issues with TCP_CORK exist for splice too, for example, for
exactly the same reasons.

And while SPLICE_F_MORE exists, it only deals with multiple splice()
calls, it doesn't deal with the "I wrote a header before I even
started using splice()" case that is the one that is mentioned for
sendfile().

Or course, technically TCP_CORK exists for plain write() use as well,
but there the portable and historical fix is simply to use writev()
and send it all in one go.

So it's hopefully only when you use sendfile() and splice() that you
end up with "oh, but I have multiple different *kinds* of sources, and
I want to cork things until I've dealt with them all".

            Linus

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Fwd: Fwd: [RFC PATCH 1/4] splice: Fix corruption of spliced data after splice() returns

[Alejandro Colomar]

[-- Attachment #1.1: Type: text/plain, Size: 3894 bytes --]

Hi Randy, Linus,

On 2023-07-20 17:24, Linus Torvalds wrote:
> Oh, and none of this discussion has had any man-page people associated with it.
> 
> So in order to maybe get the documentation clarified, let's add them.
> 
> Alejandro, Michael, see
> 
>   https://lore.kernel.org/lkml/0d10033a-7ea1-48e3-806b-f74000045915@mattwhitlock.name/
> 
> for the original email (and thread).
> 
>                  Linus

Thank you both for pointing to that!  I've been checking, and the
splice.2 source code already has some hidden comments right after a
paragraph.

(see <https://git.kernel.org/pub/scm/docs/man-pages/man-pages.git/tree/man2/splice.2#n215>)

I think we can append a paragraph like the suggested one after the
one that starts in line 216.  Can you clarify if the requirement of
taking care of not modifying the buffer applied to just splice(2) or
also tee(2) and/or vmsplice(2).  BTW, if you want to send a patch,
that's very welcome.

Cheers,
Alex

On 2023-07-20 10:13, Alejandro Colomar wrote:
> Hi Randy!
> 
> Thanks!  I'll FWD to the linux-man@ mailing list too.
> 
> Cheers,
> Alex
> 
> 
> -------- Forwarded Message --------
> Subject: Fwd: [RFC PATCH 1/4] splice: Fix corruption of spliced data after splice() returns
> Date: Wed, 19 Jul 2023 17:36:03 -0700
> From: Randy Dunlap <rdunlap@infradead.org>
> To: Alejandro Colomar <alx.manpages@gmail.com>, Michael Kerrisk <mtk.manpages@gmail.com>
> 
> FYI:
> 
> 
> -------- Forwarded Message --------
> Subject: Re: [RFC PATCH 1/4] splice: Fix corruption of spliced data after splice() returns
> Date: Wed, 19 Jul 2023 17:00:17 -0700
> From: Linus Torvalds <torvalds@linux-foundation.org>
> To: Matt Whitlock <kernel@mattwhitlock.name>
> CC: Matthew Wilcox <willy@infradead.org>, Miklos Szeredi <miklos@szeredi.hu>, David Howells <dhowells@redhat.com>, netdev@vger.kernel.org, Dave Chinner <david@fromorbit.com>, Jens Axboe <axboe@kernel.dk>, linux-fsdevel@kvack.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Christoph Hellwig <hch@lst.de>, linux-fsdevel@vger.kernel.org
> 
> On Wed, 19 Jul 2023 at 16:41, Matt Whitlock <kernel@mattwhitlock.name> wrote:
>>
>> Then that is my request. This entire complaint/discussion/argument would
>> have been avoided if splice(2) had contained a sentence like this one from
>> sendfile(2):
>>
>> "If out_fd refers to a socket or pipe with zero-copy support, callers must
>> ensure the transferred portions of the file referred to by in_fd remain
>> unmodified until the reader on the other end of out_fd has consumed the
>> transferred data."
>>
>> That is a clear warning of the perils of the implementation under the hood,
>> and it could/should be copied, more or less verbatim, to splice(2).
> 
> Ack. Internally in the kernel, the two really have always been more or
> less of intermingled.
> 
> In fact, I think splice()/sendfile()/tee() could - and maybe should -
> actually be a single man-page to make it clear that they are all
> facets of the same thing.
> 
> The issues with TCP_CORK exist for splice too, for example, for
> exactly the same reasons.
> 
> And while SPLICE_F_MORE exists, it only deals with multiple splice()
> calls, it doesn't deal with the "I wrote a header before I even
> started using splice()" case that is the one that is mentioned for
> sendfile().
> 
> Or course, technically TCP_CORK exists for plain write() use as well,
> but there the portable and historical fix is simply to use writev()
> and send it all in one go.
> 
> So it's hopefully only when you use sendfile() and splice() that you
> end up with "oh, but I have multiple different *kinds* of sources, and
> I want to cork things until I've dealt with them all".
> 
>             Linus

-- 
<http://www.alejandro-colomar.es/>
GPG key fingerprint: A9348594CE31283A826FBDD8D57633D441E25BB5


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]