Re: [PATCH] io: cork TLS writes to avoid small TLS records

[Juraj Marcin <jmarcin@redhat.com>]

Hi Daniel,

On 2026-09-09 13:00, Daniel P. Berrangé wrote:
> The migration code caches vmstate/ram writes into an iovec and
> flushes this periodically. The total amount of data to sent
> may be 100's of KB, but split across many iovec, each of which
> is potentially quite small.
> 
> The QIOChannelTLS receives the iovec, but since GNUTLS cannot
> accept iovec data, it iterates calling send for each element.
> 
> As a result of the migration data pattern, this results in
> GNUTLS putting lots of small TLS records on the wire. This
> has shown writes alternate between about 4k and 30 bytes
> in some tests.
> 
> This is triggering the nagle algorithm on migration-test for
> many of the TLS test cases, resulting in a "go slow" for I/O
> that eventually hits the migration timeout configured by the
> test.
> 
> Not every contributor reports seeing the "go slow" but for
> those who do see it, it hits >= 95% of the time for at least
> one of the migration TLS test cases run by 'make check'.
> 
> While we could disable the nagle algorithm (and multifd
> channels already do this), that would stil result in lots
> of small TLS records hitting the wire which is not good for
> throughput. The migration code flushes in batches because
> it wants large writes for high throughput.
> 
> To achieve this we must tell GNUTLS to encrypt data but not
> immediately sent TLS records by "corking" its output. Once
> the complete iovec has been written to GNUTLS, it can be
> uncorked allowing it to hit the wire.
> 
> There is added complexity with uncorking on non-blocking
> channels as not all encrypted data can be sent at once.
> qio_channel_write() will return what was sent, but some
> data might remain pending inside GNUTLS buffers. A later
> call to qio_channel_write() will provide the same plain
> text data buffers that have already been cached. Thus the
> code must attempt to uncork GNUTLS again to clear pending
> data and then deduct the equivalent amount of plain text.
> 
> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
> ---
>  crypto/tlssession.c         | 33 ++++++++++++++
>  include/crypto/tlssession.h | 25 +++++++++++
>  include/io/channel-tls.h    |  1 +
>  io/channel-tls.c            | 87 ++++++++++++++++++++++++++++++++++---
>  4 files changed, 140 insertions(+), 6 deletions(-)

I have tested the patch and after applying I could no longer reproduce
the "go slow" issue.

Thanks!

Tested-by: Juraj Marcin <jmarcin@redhat.com>