qemu-devel.nongnu.org archive mirror
 help / color / mirror / Atom feed
From: Alex Williamson <alex.williamson@redhat.com>
To: Avihai Horon <avihaih@nvidia.com>
Cc: qemu-devel@nongnu.org, "Halil Pasic" <pasic@linux.ibm.com>,
	"Christian Borntraeger" <borntraeger@linux.ibm.com>,
	"Eric Farman" <farman@linux.ibm.com>,
	"Richard Henderson" <richard.henderson@linaro.org>,
	"David Hildenbrand" <david@redhat.com>,
	"Ilya Leoshkevich" <iii@linux.ibm.com>,
	"Thomas Huth" <thuth@redhat.com>,
	"Juan Quintela" <quintela@redhat.com>,
	"Dr. David Alan Gilbert" <dgilbert@redhat.com>,
	"Michael S. Tsirkin" <mst@redhat.com>,
	"Cornelia Huck" <cohuck@redhat.com>,
	"Paolo Bonzini" <pbonzini@redhat.com>,
	"Stefan Hajnoczi" <stefanha@redhat.com>,
	"Fam Zheng" <fam@euphon.net>, "Eric Blake" <eblake@redhat.com>,
	"Vladimir Sementsov-Ogievskiy" <vsementsov@yandex-team.ru>,
	"John Snow" <jsnow@redhat.com>,
	"Cédric Le Goater" <clg@redhat.com>,
	qemu-s390x@nongnu.org, qemu-block@nongnu.org,
	"Yishai Hadas" <yishaih@nvidia.com>,
	"Jason Gunthorpe" <jgg@nvidia.com>,
	"Maor Gottlieb" <maorg@nvidia.com>,
	"Kirti Wankhede" <kwankhede@nvidia.com>,
	"Tarun Gupta" <targupta@nvidia.com>,
	"Joao Martins" <joao.m.martins@oracle.com>
Subject: Re: [PATCH v8 04/13] vfio/migration: Allow migration without VFIO IOMMU dirty tracking support
Date: Fri, 20 Jan 2023 16:06:24 -0700	[thread overview]
Message-ID: <20230120160624.2cf0762f.alex.williamson@redhat.com> (raw)
In-Reply-To: <20230116141135.12021-5-avihaih@nvidia.com>

On Mon, 16 Jan 2023 16:11:26 +0200
Avihai Horon <avihaih@nvidia.com> wrote:

> Currently, if IOMMU of a VFIO container doesn't support dirty page
> tracking, migration is blocked. This is because a DMA-able VFIO device
> can dirty RAM pages without updating QEMU about it, thus breaking the
> migration.
> 
> However, this doesn't mean that migration can't be done at all.
> In such case, allow migration and let QEMU VFIO code mark all pages
> dirty.
> 
> This guarantees that all pages that might have gotten dirty are reported
> back, and thus guarantees a valid migration even without VFIO IOMMU
> dirty tracking support.
> 
> The motivation for this patch is the introduction of iommufd [1].
> iommufd can directly implement the /dev/vfio/vfio container IOCTLs by
> mapping them into its internal ops, allowing the usage of these IOCTLs
> over iommufd. However, VFIO IOMMU dirty tracking is not supported by
> this VFIO compatibility API.
> 
> This patch will allow migration by hosts that use the VFIO compatibility
> API and prevent migration regressions caused by the lack of VFIO IOMMU
> dirty tracking support.
> 
> [1]
> https://lore.kernel.org/kvm/0-v6-a196d26f289e+11787-iommufd_jgg@nvidia.com/
> 
> Signed-off-by: Avihai Horon <avihaih@nvidia.com>
> ---
>  hw/vfio/common.c    | 20 ++++++++++++++++++--
>  hw/vfio/migration.c |  3 +--
>  2 files changed, 19 insertions(+), 4 deletions(-)
> 
> diff --git a/hw/vfio/common.c b/hw/vfio/common.c
> index 130e5d1dc7..f6dd571549 100644
> --- a/hw/vfio/common.c
> +++ b/hw/vfio/common.c
> @@ -488,6 +488,12 @@ static int vfio_dma_unmap(VFIOContainer *container,
>          return -errno;
>      }
>  
> +    if (iotlb && vfio_devices_all_running_and_saving(container)) {
> +        cpu_physical_memory_set_dirty_range(iotlb->translated_addr, size,
> +                                            tcg_enabled() ? DIRTY_CLIENTS_ALL :
> +                                            DIRTY_CLIENTS_NOCODE);

I take it this is an attempt to decipher the mask arg based on its use
in cpu_physical_memory_set_dirty_lebitmap().  I'm attempting to do the
same.  It seems like it must logically be the case that
global_dirty_tracking is set to pass the running-and-saving test, but I
can't connect the pieces.  Is this your understanding as well and the
reason we don't also need to optionally exclude DIRTY_MEMORY_MIGRATION?
Thanks,

Alex

> +    }
> +
>      return 0;
>  }
>  
> @@ -1201,6 +1207,10 @@ static void vfio_set_dirty_page_tracking(VFIOContainer *container, bool start)
>          .argsz = sizeof(dirty),
>      };
>  
> +    if (!container->dirty_pages_supported) {
> +        return;
> +    }
> +
>      if (start) {
>          dirty.flags = VFIO_IOMMU_DIRTY_PAGES_FLAG_START;
>      } else {
> @@ -1236,6 +1246,13 @@ static int vfio_get_dirty_bitmap(VFIOContainer *container, uint64_t iova,
>      uint64_t pages;
>      int ret;
>  
> +    if (!container->dirty_pages_supported) {
> +        cpu_physical_memory_set_dirty_range(ram_addr, size,
> +                                            tcg_enabled() ? DIRTY_CLIENTS_ALL :
> +                                            DIRTY_CLIENTS_NOCODE);
> +        return 0;
> +    }
> +
>      dbitmap = g_malloc0(sizeof(*dbitmap) + sizeof(*range));
>  
>      dbitmap->argsz = sizeof(*dbitmap) + sizeof(*range);
> @@ -1409,8 +1426,7 @@ static void vfio_listener_log_sync(MemoryListener *listener,
>  {
>      VFIOContainer *container = container_of(listener, VFIOContainer, listener);
>  
> -    if (vfio_listener_skipped_section(section) ||
> -        !container->dirty_pages_supported) {
> +    if (vfio_listener_skipped_section(section)) {
>          return;
>      }
>  
> diff --git a/hw/vfio/migration.c b/hw/vfio/migration.c
> index 09fe7c1de2..552c2313b2 100644
> --- a/hw/vfio/migration.c
> +++ b/hw/vfio/migration.c
> @@ -860,11 +860,10 @@ int64_t vfio_mig_bytes_transferred(void)
>  
>  int vfio_migration_probe(VFIODevice *vbasedev, Error **errp)
>  {
> -    VFIOContainer *container = vbasedev->group->container;
>      struct vfio_region_info *info = NULL;
>      int ret = -ENOTSUP;
>  
> -    if (!vbasedev->enable_migration || !container->dirty_pages_supported) {
> +    if (!vbasedev->enable_migration) {
>          goto add_blocker;
>      }
>  



  parent reply	other threads:[~2023-01-20 23:07 UTC|newest]

Thread overview: 24+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-01-16 14:11 [PATCH v8 00/13] vfio/migration: Implement VFIO migration protocol v2 Avihai Horon
2023-01-16 14:11 ` [PATCH v8 01/13] linux-headers: Update to v6.2-rc1 Avihai Horon
2023-01-16 15:38   ` Cédric Le Goater
2023-01-27 13:18   ` Michael S. Tsirkin
2023-01-16 14:11 ` [PATCH v8 02/13] migration: No save_live_pending() method uses the QEMUFile parameter Avihai Horon
2023-01-16 14:11 ` [PATCH v8 03/13] vfio/migration: Fix NULL pointer dereference bug Avihai Horon
2023-01-16 14:11 ` [PATCH v8 04/13] vfio/migration: Allow migration without VFIO IOMMU dirty tracking support Avihai Horon
2023-01-16 15:24   ` Cédric Le Goater
2023-01-20 23:06   ` Alex Williamson [this message]
2023-01-22 10:04     ` Avihai Horon
2023-01-16 14:11 ` [PATCH v8 05/13] migration/qemu-file: Add qemu_file_get_to_fd() Avihai Horon
2023-01-20 23:06   ` Alex Williamson
2023-01-16 14:11 ` [PATCH v8 06/13] vfio/common: Change vfio_devices_all_running_and_saving() logic to equivalent one Avihai Horon
2023-01-16 14:11 ` [PATCH v8 07/13] vfio/migration: Move migration v1 logic to vfio_migration_init() Avihai Horon
2023-01-16 14:11 ` [PATCH v8 08/13] vfio/migration: Rename functions/structs related to v1 protocol Avihai Horon
2023-01-16 14:11 ` [PATCH v8 09/13] vfio/migration: Implement VFIO migration protocol v2 Avihai Horon
2023-01-20 23:07   ` Alex Williamson
2023-01-22 10:31     ` Avihai Horon
2023-01-26  0:00       ` Alex Williamson
2023-01-26 13:41         ` Avihai Horon
2023-01-16 14:11 ` [PATCH v8 10/13] vfio/migration: Optimize vfio_save_pending() Avihai Horon
2023-01-16 14:11 ` [PATCH v8 11/13] vfio/migration: Remove VFIO migration protocol v1 Avihai Horon
2023-01-16 14:11 ` [PATCH v8 12/13] vfio: Alphabetize migration section of VFIO trace-events file Avihai Horon
2023-01-16 14:11 ` [PATCH v8 13/13] docs/devel: Align VFIO migration docs to v2 protocol Avihai Horon

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20230120160624.2cf0762f.alex.williamson@redhat.com \
    --to=alex.williamson@redhat.com \
    --cc=avihaih@nvidia.com \
    --cc=borntraeger@linux.ibm.com \
    --cc=clg@redhat.com \
    --cc=cohuck@redhat.com \
    --cc=david@redhat.com \
    --cc=dgilbert@redhat.com \
    --cc=eblake@redhat.com \
    --cc=fam@euphon.net \
    --cc=farman@linux.ibm.com \
    --cc=iii@linux.ibm.com \
    --cc=jgg@nvidia.com \
    --cc=joao.m.martins@oracle.com \
    --cc=jsnow@redhat.com \
    --cc=kwankhede@nvidia.com \
    --cc=maorg@nvidia.com \
    --cc=mst@redhat.com \
    --cc=pasic@linux.ibm.com \
    --cc=pbonzini@redhat.com \
    --cc=qemu-block@nongnu.org \
    --cc=qemu-devel@nongnu.org \
    --cc=qemu-s390x@nongnu.org \
    --cc=quintela@redhat.com \
    --cc=richard.henderson@linaro.org \
    --cc=stefanha@redhat.com \
    --cc=targupta@nvidia.com \
    --cc=thuth@redhat.com \
    --cc=vsementsov@yandex-team.ru \
    --cc=yishaih@nvidia.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).