From: "Daniel P. Berrangé" <berrange@redhat.com>
To: qemu-devel@nongnu.org
Cc: "Paolo Bonzini" <pbonzini@redhat.com>,
"Daniel P. Berrangé" <berrange@redhat.com>,
"Peter Xu" <peterx@redhat.com>,
"Philippe Mathieu-Daudé" <philmd@linaro.org>,
"Marc-André Lureau" <marcandre.lureau@redhat.com>,
"Fabiano Rosas" <farosas@suse.de>
Subject: [PATCH 0/4] migration: workaround GNUTLS live migration crashes
Date: Fri, 18 Jul 2025 16:05:10 +0100 [thread overview]
Message-ID: <20250718150514.2635338-1-berrange@redhat.com> (raw)
TL:DR: GNUTLS is liable to crash QEMU when live migration is run
with TLS enabled and a return path channel is present, if approx
64 GB of data is transferred. This is easily triggered in a 16 GB
VM with 4 CPUs, by running 'stress-ng --vm 4 --vm-bytes 80%' to
prevent convergance until 64 GB of RAM has been copied. Then
triggering post-copy switchover, or removing the stress workload
to allow completion, will crash it.
The only live migration scenario that should avoid this danger
is multifd, since the high volume data transfers are handled in
dedicated TCP connections which are unidirectional. The main
bi-directionl TCP connection is only for co-ordination purposes
This patch implements a workaround that will prevent future QEMU
versions from triggering the crash.
The only way to avoid the crash with *existing* running QEMU
processes is to change the TLS cipher priority string to avoid
use of AES with TLS 1.3. This can be done with the 'priority'
field in the 'tls-creds-x509' object.eg
-object tls-creds-x509,id=tls0,priority=NORMAL:-AES-256-GCM:-AES-128-GCM:-AES-128-CCM
which should force the use of CHACHA20-POLY1305 which does not
require TLS re-keying after 16 million sent records (64 GB of
migration data).
https://gitlab.com/qemu-project/qemu/-/issues/1937
On RHEL/Fedora distros you can also use the system wide crypto
priorities to override this from the migration *target* host
by creating /etc/crypto-policies/local.d/gnutls-qemu.config
containing
QEMU=NONE:+ECDHE-RSA:+ECDHE-ECDSA:+RSA:+DHE-RSA:+GROUP-X25519:+GROUP-X448:+GROUP-SECP256R1:+GROUP-SECP384R1:+GROUP-SECP521R1:+GROUP-FF
and running 'update-crypto-policies'. I recommend the QEMU
level 'tls-creds-x509' workaround though, which new libvirt
patches can soon do:
https://lists.libvirt.org/archives/list/devel@lists.libvirt.org/thread/LX5KMIUFZSP5DPUXKJDFYBZI5TIE3E5N/
Daniel P. Berrangé (4):
crypto: implement workaround for GNUTLS thread safety problems
io: add support for activating TLS thread safety workaround
migration: activate TLS thread safety workaround
crypto: add tracing & warning about GNUTLS countermeasures
crypto/tlssession.c | 99 +++++++++++++++++++++++++++++++++--
crypto/trace-events | 2 +
include/crypto/tlssession.h | 14 +++++
include/io/channel.h | 1 +
io/channel-tls.c | 5 ++
meson.build | 9 ++++
meson_options.txt | 2 +
migration/tls.c | 9 ++++
scripts/meson-buildoptions.sh | 5 ++
9 files changed, 143 insertions(+), 3 deletions(-)
--
2.50.1
next reply other threads:[~2025-07-18 15:15 UTC|newest]
Thread overview: 17+ messages / expand[flat|nested] mbox.gz Atom feed top
2025-07-18 15:05 Daniel P. Berrangé [this message]
2025-07-18 15:05 ` [PATCH 1/4] crypto: implement workaround for GNUTLS thread safety problems Daniel P. Berrangé
2025-07-21 14:52 ` Fabiano Rosas
2025-07-21 19:19 ` Fabiano Rosas
2025-07-18 15:05 ` [PATCH 2/4] io: add support for activating TLS thread safety workaround Daniel P. Berrangé
2025-07-21 14:52 ` Fabiano Rosas
2025-07-18 15:05 ` [PATCH 3/4] migration: activate " Daniel P. Berrangé
2025-07-21 14:52 ` Fabiano Rosas
2025-07-18 15:05 ` [PATCH 4/4] crypto: add tracing & warning about GNUTLS countermeasures Daniel P. Berrangé
2025-07-21 14:52 ` Fabiano Rosas
2025-07-21 19:32 ` Fabiano Rosas
2025-07-21 14:56 ` [PATCH 0/4] migration: workaround GNUTLS live migration crashes Fabiano Rosas
2025-07-21 15:03 ` Daniel P. Berrangé
2025-07-21 15:14 ` Fabiano Rosas
2025-07-21 15:28 ` Daniel P. Berrangé
2025-07-26 6:24 ` Michael Tokarev
2025-07-28 9:04 ` Daniel P. Berrangé
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20250718150514.2635338-1-berrange@redhat.com \
--to=berrange@redhat.com \
--cc=farosas@suse.de \
--cc=marcandre.lureau@redhat.com \
--cc=pbonzini@redhat.com \
--cc=peterx@redhat.com \
--cc=philmd@linaro.org \
--cc=qemu-devel@nongnu.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).