Re: [PATCH 0/6] tcp: support preloading data on a listening socket

[Jeremy Harris <jgh@exim.org>]

Hi Neal,

Thanks for the initial review.


On 2025/05/16 7:19 PM, Neal Cardwell wrote:
> On Fri, May 16, 2025 at 11:55 AM Jeremy Harris <jgh@exim.org> wrote:
>>
>> Support write to a listen TCP socket, for immediate
>> transmission on passive connection establishments.
>>
>> On a normal connection transmission is triggered by the receipt of
>> the 3rd-ack. On a fastopen (with accepted cookie) connection the data
>> is sent in the synack packet.
>>
>> The data preload is done using a sendmsg with a newly-defined flag
>> (MSG_PRELOAD); the amount of data limited to a single linear sk_buff.
>> Note that this definition is the last-but-two bit available if "int"
>> is 32 bits.
> 
> Can you please add a bit more context, like:
> 
> + What is the motivating use case? (Accelerating Exim?)

Accelerating any server-first ULP, SMTP being the major use I
know of (and yes, Exim is my primary testcase and is operational
against a test kernel with this patch series).

One caveat: the initial server data cannot change from one passive
connection to another.

> Is this
> targeted for connections using encryption (like TLS/SSL), or just
> plain-text connections?

TLS-on-connect cannot benefit, being client-first.  SMTP that uses
STARTTLS can take advantage of it, as can plaintext SMTP.

I would not expect https to be able to use it.


> + What are the exact performance improvements you are seeing in your
> benchmarks that (a) motivate this, and (b) justify any performance
> impact on the TCP stack?

Because of the lack of userland roundtrip needed for the initial server
data, there is a latency benefit.  This is better for the TFO-C case,
but also significant for the non-TFO case.

Packet capture (laptop, loopback, TFO-C case) for initial SYN to first
client data packet (5 samples):

- baseline   TFO_C      1064 1470 1455 1547 1595  usec
- patched    non-TFO     140  150  159  144  153  usec
- patched    TFO_C       142  149  149  125  125  usec



One fewer packet is sent by the server in most packet captures, sometimes
one fewer in each direction.  There is one less application kernel entry/exit
on the server.

I'm hoping those differences will add up to both less cpu time (on both
endpoints) and less wire-time.  However, I have not run benchmarks looking
for a change in peak rate of connection-handling.



In summary, this is the mirror of TCP Fast Open client data: the latency
benefit is probably the most useful aspect.


> + Regarding "Support write to a listen TCP socket, for immediate
> transmission on passive connection establishments.": can you please
> make it explicitly clear whether the data written to the listening
> socket is saved and transmitted on all future successful passive
> sockets that are created for the listener,

This.  The data is copied for each future passive socket from this
listener,

> or is just transmitted on
> the next connection that is created?

(and not this option).




I'll copy these comments in any future v2.
As Eric says, I should run KASAN/syzbot first.

-- 
Cheers,
   Jeremy