qemu-devel.nongnu.org archive mirror
 help / color / mirror / Atom feed
From: "Daniel P. Berrangé" <berrange@redhat.com>
To: qemu-devel@nongnu.org
Cc: "Paolo Bonzini" <pbonzini@redhat.com>,
	"Daniel P. Berrangé" <berrange@redhat.com>,
	"Peter Xu" <peterx@redhat.com>,
	"Philippe Mathieu-Daudé" <philmd@linaro.org>,
	"Marc-André Lureau" <marcandre.lureau@redhat.com>,
	"Fabiano Rosas" <farosas@suse.de>
Subject: [PATCH 0/4] migration: workaround GNUTLS live migration crashes
Date: Fri, 18 Jul 2025 16:05:10 +0100	[thread overview]
Message-ID: <20250718150514.2635338-1-berrange@redhat.com> (raw)

TL:DR: GNUTLS is liable to crash QEMU when live migration is run
with TLS enabled and a return path channel is present, if approx
64 GB of data is transferred. This is easily triggered in a 16 GB
VM with 4 CPUs, by running 'stress-ng --vm 4 --vm-bytes 80%' to
prevent convergance until 64 GB of RAM has been copied. Then
triggering post-copy switchover, or removing the stress workload
to allow completion, will crash it.

The only live migration scenario that should avoid this danger
is multifd, since the high volume data transfers are handled in
dedicated TCP connections which are unidirectional. The main
bi-directionl TCP connection is only for co-ordination purposes

This patch implements a workaround that will prevent future QEMU
versions from triggering the crash.

The only way to avoid the crash with *existing* running QEMU
processes is to change the TLS cipher priority string to avoid
use of AES with TLS 1.3. This can be done with the 'priority'
field in the 'tls-creds-x509' object.eg

  -object tls-creds-x509,id=tls0,priority=NORMAL:-AES-256-GCM:-AES-128-GCM:-AES-128-CCM

which should force the use of CHACHA20-POLY1305 which does not
require TLS re-keying after 16 million sent records (64 GB of
migration data).

  https://gitlab.com/qemu-project/qemu/-/issues/1937

On RHEL/Fedora distros you can also use the system wide crypto
priorities to override this from the migration *target* host
by creating /etc/crypto-policies/local.d/gnutls-qemu.config
containing

  QEMU=NONE:+ECDHE-RSA:+ECDHE-ECDSA:+RSA:+DHE-RSA:+GROUP-X25519:+GROUP-X448:+GROUP-SECP256R1:+GROUP-SECP384R1:+GROUP-SECP521R1:+GROUP-FF

and running 'update-crypto-policies'. I recommend the QEMU
level 'tls-creds-x509' workaround though, which new libvirt
patches can soon do:

  https://lists.libvirt.org/archives/list/devel@lists.libvirt.org/thread/LX5KMIUFZSP5DPUXKJDFYBZI5TIE3E5N/

Daniel P. Berrangé (4):
  crypto: implement workaround for GNUTLS thread safety problems
  io: add support for activating TLS thread safety workaround
  migration: activate TLS thread safety workaround
  crypto: add tracing & warning about GNUTLS countermeasures

 crypto/tlssession.c           | 99 +++++++++++++++++++++++++++++++++--
 crypto/trace-events           |  2 +
 include/crypto/tlssession.h   | 14 +++++
 include/io/channel.h          |  1 +
 io/channel-tls.c              |  5 ++
 meson.build                   |  9 ++++
 meson_options.txt             |  2 +
 migration/tls.c               |  9 ++++
 scripts/meson-buildoptions.sh |  5 ++
 9 files changed, 143 insertions(+), 3 deletions(-)

-- 
2.50.1



             reply	other threads:[~2025-07-18 15:15 UTC|newest]

Thread overview: 17+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2025-07-18 15:05 Daniel P. Berrangé [this message]
2025-07-18 15:05 ` [PATCH 1/4] crypto: implement workaround for GNUTLS thread safety problems Daniel P. Berrangé
2025-07-21 14:52   ` Fabiano Rosas
2025-07-21 19:19   ` Fabiano Rosas
2025-07-18 15:05 ` [PATCH 2/4] io: add support for activating TLS thread safety workaround Daniel P. Berrangé
2025-07-21 14:52   ` Fabiano Rosas
2025-07-18 15:05 ` [PATCH 3/4] migration: activate " Daniel P. Berrangé
2025-07-21 14:52   ` Fabiano Rosas
2025-07-18 15:05 ` [PATCH 4/4] crypto: add tracing & warning about GNUTLS countermeasures Daniel P. Berrangé
2025-07-21 14:52   ` Fabiano Rosas
2025-07-21 19:32   ` Fabiano Rosas
2025-07-21 14:56 ` [PATCH 0/4] migration: workaround GNUTLS live migration crashes Fabiano Rosas
2025-07-21 15:03   ` Daniel P. Berrangé
2025-07-21 15:14     ` Fabiano Rosas
2025-07-21 15:28       ` Daniel P. Berrangé
2025-07-26  6:24 ` Michael Tokarev
2025-07-28  9:04   ` Daniel P. Berrangé

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20250718150514.2635338-1-berrange@redhat.com \
    --to=berrange@redhat.com \
    --cc=farosas@suse.de \
    --cc=marcandre.lureau@redhat.com \
    --cc=pbonzini@redhat.com \
    --cc=peterx@redhat.com \
    --cc=philmd@linaro.org \
    --cc=qemu-devel@nongnu.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).