From: Alex Williamson <alex.williamson@redhat.com>
To: Avihai Horon <avihaih@nvidia.com>
Cc: qemu-devel@nongnu.org, "Halil Pasic" <pasic@linux.ibm.com>,
"Christian Borntraeger" <borntraeger@linux.ibm.com>,
"Eric Farman" <farman@linux.ibm.com>,
"Richard Henderson" <richard.henderson@linaro.org>,
"David Hildenbrand" <david@redhat.com>,
"Ilya Leoshkevich" <iii@linux.ibm.com>,
"Thomas Huth" <thuth@redhat.com>,
"Juan Quintela" <quintela@redhat.com>,
"Dr. David Alan Gilbert" <dgilbert@redhat.com>,
"Michael S. Tsirkin" <mst@redhat.com>,
"Cornelia Huck" <cohuck@redhat.com>,
"Paolo Bonzini" <pbonzini@redhat.com>,
"Stefan Hajnoczi" <stefanha@redhat.com>,
"Fam Zheng" <fam@euphon.net>, "Eric Blake" <eblake@redhat.com>,
"Vladimir Sementsov-Ogievskiy" <vsementsov@yandex-team.ru>,
"John Snow" <jsnow@redhat.com>,
"Cédric Le Goater" <clg@redhat.com>,
qemu-s390x@nongnu.org, qemu-block@nongnu.org,
"Yishai Hadas" <yishaih@nvidia.com>,
"Jason Gunthorpe" <jgg@nvidia.com>,
"Maor Gottlieb" <maorg@nvidia.com>,
"Kirti Wankhede" <kwankhede@nvidia.com>,
"Tarun Gupta" <targupta@nvidia.com>,
"Joao Martins" <joao.m.martins@oracle.com>
Subject: Re: [PATCH v8 04/13] vfio/migration: Allow migration without VFIO IOMMU dirty tracking support
Date: Fri, 20 Jan 2023 16:06:24 -0700 [thread overview]
Message-ID: <20230120160624.2cf0762f.alex.williamson@redhat.com> (raw)
In-Reply-To: <20230116141135.12021-5-avihaih@nvidia.com>
On Mon, 16 Jan 2023 16:11:26 +0200
Avihai Horon <avihaih@nvidia.com> wrote:
> Currently, if IOMMU of a VFIO container doesn't support dirty page
> tracking, migration is blocked. This is because a DMA-able VFIO device
> can dirty RAM pages without updating QEMU about it, thus breaking the
> migration.
>
> However, this doesn't mean that migration can't be done at all.
> In such case, allow migration and let QEMU VFIO code mark all pages
> dirty.
>
> This guarantees that all pages that might have gotten dirty are reported
> back, and thus guarantees a valid migration even without VFIO IOMMU
> dirty tracking support.
>
> The motivation for this patch is the introduction of iommufd [1].
> iommufd can directly implement the /dev/vfio/vfio container IOCTLs by
> mapping them into its internal ops, allowing the usage of these IOCTLs
> over iommufd. However, VFIO IOMMU dirty tracking is not supported by
> this VFIO compatibility API.
>
> This patch will allow migration by hosts that use the VFIO compatibility
> API and prevent migration regressions caused by the lack of VFIO IOMMU
> dirty tracking support.
>
> [1]
> https://lore.kernel.org/kvm/0-v6-a196d26f289e+11787-iommufd_jgg@nvidia.com/
>
> Signed-off-by: Avihai Horon <avihaih@nvidia.com>
> ---
> hw/vfio/common.c | 20 ++++++++++++++++++--
> hw/vfio/migration.c | 3 +--
> 2 files changed, 19 insertions(+), 4 deletions(-)
>
> diff --git a/hw/vfio/common.c b/hw/vfio/common.c
> index 130e5d1dc7..f6dd571549 100644
> --- a/hw/vfio/common.c
> +++ b/hw/vfio/common.c
> @@ -488,6 +488,12 @@ static int vfio_dma_unmap(VFIOContainer *container,
> return -errno;
> }
>
> + if (iotlb && vfio_devices_all_running_and_saving(container)) {
> + cpu_physical_memory_set_dirty_range(iotlb->translated_addr, size,
> + tcg_enabled() ? DIRTY_CLIENTS_ALL :
> + DIRTY_CLIENTS_NOCODE);
I take it this is an attempt to decipher the mask arg based on its use
in cpu_physical_memory_set_dirty_lebitmap(). I'm attempting to do the
same. It seems like it must logically be the case that
global_dirty_tracking is set to pass the running-and-saving test, but I
can't connect the pieces. Is this your understanding as well and the
reason we don't also need to optionally exclude DIRTY_MEMORY_MIGRATION?
Thanks,
Alex
> + }
> +
> return 0;
> }
>
> @@ -1201,6 +1207,10 @@ static void vfio_set_dirty_page_tracking(VFIOContainer *container, bool start)
> .argsz = sizeof(dirty),
> };
>
> + if (!container->dirty_pages_supported) {
> + return;
> + }
> +
> if (start) {
> dirty.flags = VFIO_IOMMU_DIRTY_PAGES_FLAG_START;
> } else {
> @@ -1236,6 +1246,13 @@ static int vfio_get_dirty_bitmap(VFIOContainer *container, uint64_t iova,
> uint64_t pages;
> int ret;
>
> + if (!container->dirty_pages_supported) {
> + cpu_physical_memory_set_dirty_range(ram_addr, size,
> + tcg_enabled() ? DIRTY_CLIENTS_ALL :
> + DIRTY_CLIENTS_NOCODE);
> + return 0;
> + }
> +
> dbitmap = g_malloc0(sizeof(*dbitmap) + sizeof(*range));
>
> dbitmap->argsz = sizeof(*dbitmap) + sizeof(*range);
> @@ -1409,8 +1426,7 @@ static void vfio_listener_log_sync(MemoryListener *listener,
> {
> VFIOContainer *container = container_of(listener, VFIOContainer, listener);
>
> - if (vfio_listener_skipped_section(section) ||
> - !container->dirty_pages_supported) {
> + if (vfio_listener_skipped_section(section)) {
> return;
> }
>
> diff --git a/hw/vfio/migration.c b/hw/vfio/migration.c
> index 09fe7c1de2..552c2313b2 100644
> --- a/hw/vfio/migration.c
> +++ b/hw/vfio/migration.c
> @@ -860,11 +860,10 @@ int64_t vfio_mig_bytes_transferred(void)
>
> int vfio_migration_probe(VFIODevice *vbasedev, Error **errp)
> {
> - VFIOContainer *container = vbasedev->group->container;
> struct vfio_region_info *info = NULL;
> int ret = -ENOTSUP;
>
> - if (!vbasedev->enable_migration || !container->dirty_pages_supported) {
> + if (!vbasedev->enable_migration) {
> goto add_blocker;
> }
>
next prev parent reply other threads:[~2023-01-20 23:07 UTC|newest]
Thread overview: 24+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-01-16 14:11 [PATCH v8 00/13] vfio/migration: Implement VFIO migration protocol v2 Avihai Horon
2023-01-16 14:11 ` [PATCH v8 01/13] linux-headers: Update to v6.2-rc1 Avihai Horon
2023-01-16 15:38 ` Cédric Le Goater
2023-01-27 13:18 ` Michael S. Tsirkin
2023-01-16 14:11 ` [PATCH v8 02/13] migration: No save_live_pending() method uses the QEMUFile parameter Avihai Horon
2023-01-16 14:11 ` [PATCH v8 03/13] vfio/migration: Fix NULL pointer dereference bug Avihai Horon
2023-01-16 14:11 ` [PATCH v8 04/13] vfio/migration: Allow migration without VFIO IOMMU dirty tracking support Avihai Horon
2023-01-16 15:24 ` Cédric Le Goater
2023-01-20 23:06 ` Alex Williamson [this message]
2023-01-22 10:04 ` Avihai Horon
2023-01-16 14:11 ` [PATCH v8 05/13] migration/qemu-file: Add qemu_file_get_to_fd() Avihai Horon
2023-01-20 23:06 ` Alex Williamson
2023-01-16 14:11 ` [PATCH v8 06/13] vfio/common: Change vfio_devices_all_running_and_saving() logic to equivalent one Avihai Horon
2023-01-16 14:11 ` [PATCH v8 07/13] vfio/migration: Move migration v1 logic to vfio_migration_init() Avihai Horon
2023-01-16 14:11 ` [PATCH v8 08/13] vfio/migration: Rename functions/structs related to v1 protocol Avihai Horon
2023-01-16 14:11 ` [PATCH v8 09/13] vfio/migration: Implement VFIO migration protocol v2 Avihai Horon
2023-01-20 23:07 ` Alex Williamson
2023-01-22 10:31 ` Avihai Horon
2023-01-26 0:00 ` Alex Williamson
2023-01-26 13:41 ` Avihai Horon
2023-01-16 14:11 ` [PATCH v8 10/13] vfio/migration: Optimize vfio_save_pending() Avihai Horon
2023-01-16 14:11 ` [PATCH v8 11/13] vfio/migration: Remove VFIO migration protocol v1 Avihai Horon
2023-01-16 14:11 ` [PATCH v8 12/13] vfio: Alphabetize migration section of VFIO trace-events file Avihai Horon
2023-01-16 14:11 ` [PATCH v8 13/13] docs/devel: Align VFIO migration docs to v2 protocol Avihai Horon
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20230120160624.2cf0762f.alex.williamson@redhat.com \
--to=alex.williamson@redhat.com \
--cc=avihaih@nvidia.com \
--cc=borntraeger@linux.ibm.com \
--cc=clg@redhat.com \
--cc=cohuck@redhat.com \
--cc=david@redhat.com \
--cc=dgilbert@redhat.com \
--cc=eblake@redhat.com \
--cc=fam@euphon.net \
--cc=farman@linux.ibm.com \
--cc=iii@linux.ibm.com \
--cc=jgg@nvidia.com \
--cc=joao.m.martins@oracle.com \
--cc=jsnow@redhat.com \
--cc=kwankhede@nvidia.com \
--cc=maorg@nvidia.com \
--cc=mst@redhat.com \
--cc=pasic@linux.ibm.com \
--cc=pbonzini@redhat.com \
--cc=qemu-block@nongnu.org \
--cc=qemu-devel@nongnu.org \
--cc=qemu-s390x@nongnu.org \
--cc=quintela@redhat.com \
--cc=richard.henderson@linaro.org \
--cc=stefanha@redhat.com \
--cc=targupta@nvidia.com \
--cc=thuth@redhat.com \
--cc=vsementsov@yandex-team.ru \
--cc=yishaih@nvidia.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).