From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.5 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6606FC43603 for ; Wed, 11 Dec 2019 13:47:17 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 2F5A920836 for ; Wed, 11 Dec 2019 13:47:17 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="DYbeXd5C" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2F5A920836 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:42690 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1if2Kg-0001Ye-Ho for qemu-devel@archiver.kernel.org; Wed, 11 Dec 2019 08:47:16 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:50365) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1if2Is-0000NN-M4 for qemu-devel@nongnu.org; Wed, 11 Dec 2019 08:45:24 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1if2Ip-0002M1-IC for qemu-devel@nongnu.org; Wed, 11 Dec 2019 08:45:20 -0500 Received: from us-smtp-2.mimecast.com ([207.211.31.81]:24070 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1if2Ip-0002K0-2z for qemu-devel@nongnu.org; Wed, 11 Dec 2019 08:45:19 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1576071917; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=1MX3ocqx0r4QRee3SKIz1dZ85rbV1iZlW2IFbvxZKyM=; b=DYbeXd5CEY71P8eGbv/ePh4TGgXLV4Rm4q7KHIdKRHlbUnm5UZ89SP9PPcgtzb1ocYesLX N5I7GK+XbwlNpDVCOTDZN1El7z8Y3VX1Mt0/ii2gx/IHLAUN7U8ukj2uON4QmGrKgSjU3u nEOfXVa8roMAWlONojH98+EPFncJOaM= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-415-VYdXHUw9M0G9x4a9YuutIw-1; Wed, 11 Dec 2019 08:45:16 -0500 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 7A4BE8E5453 for ; Wed, 11 Dec 2019 13:45:15 +0000 (UTC) Received: from localhost (ovpn-112-63.ams2.redhat.com [10.36.112.63]) by smtp.corp.redhat.com (Postfix) with ESMTP id 50AEA6013D; Wed, 11 Dec 2019 13:45:10 +0000 (UTC) From: =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= To: qemu-devel@nongnu.org Subject: [PATCH v6 0/8] Add dbus-vmstate Date: Wed, 11 Dec 2019 17:44:58 +0400 Message-Id: <20191211134506.1803403-1-marcandre.lureau@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-MC-Unique: VYdXHUw9M0G9x4a9YuutIw-1 X-Mimecast-Spam-Score: 0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 207.211.31.81 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: berrange@redhat.com, quintela@redhat.com, mprivozn@redhat.com, dgilbert@redhat.com, =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , pbonzini@redhat.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Hi, With external processes or helpers participating to the VM support, it becomes necessary to handle their migration. Various options exist to transfer their state: 1) as the VM memory, RAM or devices (we could say that's how vhost-user devices can be handled today, they are expected to restore from ring state) 2) other "vmstate" (as with TPM emulator state blobs) 3) left to be handled by management layer 1) is not practical, since an external processes may legitimatelly need arbitrary state date to back a device or a service, or may not even have an associated device. 2) needs ad-hoc code for each helper, but is simple and working 3) is complicated for management layer, QEMU has the migration timing The proposed "dbus-vmstate" object will connect to a given D-Bus address, and save/load from org.qemu.VMState1 owners on migration. Thus helpers can easily have their state migrated with QEMU, without implementing ad-hoc support (such as done for TPM emulation) D-Bus is ubiquitous on Linux (it is systemd IPC), and can be made to work on various other OSes. There are several implementations and good bindings for various languages. (the tests/dbus-vmstate-test.c is a good example of how simple the implementation of services can be, even in C) dbus-vmstate is put into use by the libvirt series "[PATCH 00/23] Use a slirp helper process". v6: - rebased (minor change in patch 2) v5: - trying to fix patchew/ci: install dbus-daemon in containers, skip test if unavailable v4: - add Daniel security scenarios to the D-Bus document - misc doc improvements - add "util: add dbus helper unit" patch, with qemu_dbus_get_queued_owners() - add "configure: add GDBUS_CODEGEN", explaining why gio-unix is required when available - silence the expected failing tests - update copyright headers, MAINTAINERS - add r-b/a-b tags - rebased (Note: patchew dbus test fails for unclear reasons, but I can't reproduce locally nor on travis) v3: - after various discussions on helper processes, we settled on a preference for having a bus for communications. This version is actually v1 updated. - added a dbus.rst document to describe D-Bus recommendations for QEMU - added dbus-vmstate-daemon.sh to play with the dbus-daemon configuration (although it is not very useful in the context of a single UID) - added a new vmstate interface, so that any object can implement VMStateDescription, and converted dbus-vmstate - added "migration: fix vmdesc leak on vmstate_save() error" - convert to g_auto v2: - D-Bus is most common and practical through a bus, but it requires a daemon to be running. I argue that the benefits outweight the cost of running an extra daemon in v1 in the context of multi-process qemu, but it is also possible to connect in p2p mode as done in this new version. Marc-Andr=C3=A9 Lureau (8): vmstate: add qom interface to get id vmstate: replace DeviceState with VMStateIf docs: start a document to describe D-Bus usage util: add dbus helper unit Add dbus-vmstate object configure: add GDBUS_CODEGEN dockerfiles: add dbus-daemon to some of latest distributions tests: add dbus-vmstate-test MAINTAINERS | 12 + backends/Makefile.objs | 4 + backends/dbus-vmstate.c | 496 +++++++++++++++++++++++ configure | 7 + docs/interop/dbus-vmstate.rst | 74 ++++ docs/interop/dbus.rst | 104 +++++ docs/interop/index.rst | 2 + hw/block/onenand.c | 2 +- hw/core/Makefile.objs | 1 + hw/core/qdev.c | 21 +- hw/core/vmstate-if.c | 23 ++ hw/ide/cmd646.c | 2 +- hw/ide/isa.c | 2 +- hw/ide/piix.c | 2 +- hw/ide/via.c | 2 +- hw/misc/max111x.c | 2 +- hw/net/eepro100.c | 4 +- hw/net/virtio-net.c | 3 +- hw/nvram/eeprom93xx.c | 4 +- hw/ppc/spapr_drc.c | 9 +- hw/ppc/spapr_iommu.c | 4 +- hw/s390x/s390-skeys.c | 2 +- include/hw/vmstate-if.h | 40 ++ include/migration/register.h | 4 +- include/migration/vmstate.h | 10 +- include/qemu/dbus.h | 18 + migration/savevm.c | 20 +- stubs/vmstate.c | 4 +- tests/Makefile.include | 23 +- tests/dbus-vmstate-daemon.sh | 95 +++++ tests/dbus-vmstate-test.c | 399 ++++++++++++++++++ tests/dbus-vmstate1.xml | 12 + tests/docker/dockerfiles/centos7.docker | 1 + tests/docker/dockerfiles/debian10.docker | 1 + tests/docker/dockerfiles/fedora.docker | 1 + tests/docker/dockerfiles/ubuntu.docker | 1 + util/Makefile.objs | 3 + util/dbus.c | 55 +++ 38 files changed, 1430 insertions(+), 39 deletions(-) create mode 100644 backends/dbus-vmstate.c create mode 100644 docs/interop/dbus-vmstate.rst create mode 100644 docs/interop/dbus.rst create mode 100644 hw/core/vmstate-if.c create mode 100644 include/hw/vmstate-if.h create mode 100644 include/qemu/dbus.h create mode 100755 tests/dbus-vmstate-daemon.sh create mode 100644 tests/dbus-vmstate-test.c create mode 100644 tests/dbus-vmstate1.xml create mode 100644 util/dbus.c --=20 2.24.0.308.g228f53135a