Re: [PATCH v3 00/13] vfio/migration: Device dirty page tracking

[Joao Martins <joao.m.martins@oracle.com>]

[-- Attachment #1: Type: text/plain, Size: 5373 bytes --]

On 05/03/2023 20:57, Alex Williamson wrote:
> On Sat,  4 Mar 2023 01:43:30 +0000
> Joao Martins <joao.m.martins@oracle.com> wrote:
> 
>> Hey,
>>
>> Presented herewith a series based on the basic VFIO migration protocol v2
>> implementation [1].
>>
>> It is split from its parent series[5] to solely focus on device dirty
>> page tracking. Device dirty page tracking allows the VFIO device to
>> record its DMAs and report them back when needed. This is part of VFIO
>> migration and is used during pre-copy phase of migration to track the
>> RAM pages that the device has written to and mark those pages dirty, so
>> they can later be re-sent to target.
>>
>> Device dirty page tracking uses the DMA logging uAPI to discover device
>> capabilities, to start and stop tracking, and to get dirty page bitmap
>> report. Extra details and uAPI definition can be found here [3].
>>
>> Device dirty page tracking operates in VFIOContainer scope. I.e., When
>> dirty tracking is started, stopped or dirty page report is queried, all
>> devices within a VFIOContainer are iterated and for each of them device
>> dirty page tracking is started, stopped or dirty page report is queried,
>> respectively.
>>
>> Device dirty page tracking is used only if all devices within a
>> VFIOContainer support it. Otherwise, VFIO IOMMU dirty page tracking is
>> used, and if that is not supported as well, memory is perpetually marked
>> dirty by QEMU. Note that since VFIO IOMMU dirty page tracking has no HW
>> support, the last two usually have the same effect of perpetually
>> marking all pages dirty.
>>
>> Normally, when asked to start dirty tracking, all the currently DMA
>> mapped ranges are tracked by device dirty page tracking. If using a
>> vIOMMU we block live migration. It's temporary and a separate series is
>> going to add support for it. Thus this series focus on getting the
>> ground work first.
>>
>> The series is organized as follows:
>>
>> - Patches 1-7: Fix bugs and do some preparatory work required prior to
>>   adding device dirty page tracking.
>> - Patches 8-10: Implement device dirty page tracking.
>> - Patch 11: Blocks live migration with vIOMMU.
>> - Patches 12-13 enable device dirty page tracking and document it.
>>
>> Comments, improvements as usual appreciated.
> 
> Still some CI failures:
> 
> https://gitlab.com/alex.williamson/qemu/-/pipelines/796657474
> 
> The docker failures are normal, afaict the rest are not.  Thanks,
> 

Ugh, sorry

The patch below scissors mark (and also attached as a file) fixes those build
issues. I managed to reproduce on i386 target builds, and these changes fix my
32-bit build.

I don't have a working Gitlab setup[*] though to trigger the CI to enable to
wealth of targets it build-tests. If you could kindly test the patch attached in
a new pipeline (applied on top of the branch you just build) below to understand
if the CI gets happy. I will include these changes in the right patches (patch 8
and 10) for the v4 spin.

	Joao

[*] I'm working with Gitlab support to understand what's wrong there with my
account.

----------------->8-----------------

From bbf2c3bbb9c9e97f12dfe49f85dac8cc1f0c5d97 Mon Sep 17 00:00:00 2001
From: Joao Martins <joao.m.martins@oracle.com>
Date: Sun, 5 Mar 2023 18:12:29 -0500
Subject: [PATCH v3 14/13] vfio/common: Fix 32-bit builds

Signed-off-by: Joao Martins <joao.m.martins@oracle.com>
---
 hw/vfio/common.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/hw/vfio/common.c b/hw/vfio/common.c
index 9b909f856722..eecff5bb16c6 100644
--- a/hw/vfio/common.c
+++ b/hw/vfio/common.c
@@ -1554,7 +1554,7 @@ vfio_device_feature_dma_logging_start_create(VFIOContainer
*container)
         return NULL;
     }

-    control->ranges = (__aligned_u64)ranges;
+    control->ranges = (__u64)(uintptr_t)ranges;
     if (tracking->max32) {
         ranges->iova = tracking->min32;
         ranges->length = (tracking->max32 - tracking->min32) + 1;
@@ -1578,7 +1578,7 @@ static void vfio_device_feature_dma_logging_start_destroy(
     struct vfio_device_feature_dma_logging_control *control =
         (struct vfio_device_feature_dma_logging_control *)feature->data;
     struct vfio_device_feature_dma_logging_range *ranges =
-        (struct vfio_device_feature_dma_logging_range *)control->ranges;
+        (struct vfio_device_feature_dma_logging_range *)(uintptr_t)
control->ranges;

     g_free(ranges);
     g_free(feature);
@@ -1646,7 +1646,7 @@ static int vfio_device_dma_logging_report(VFIODevice
*vbasedev, hwaddr iova,
 {
     uint64_t buf[DIV_ROUND_UP(sizeof(struct vfio_device_feature) +
                         sizeof(struct vfio_device_feature_dma_logging_report),
-                        sizeof(__aligned_u64))] = {};
+                        sizeof(__u64))] = {};
     struct vfio_device_feature *feature = (struct vfio_device_feature *)buf;
     struct vfio_device_feature_dma_logging_report *report =
         (struct vfio_device_feature_dma_logging_report *)feature->data;
@@ -1654,7 +1654,7 @@ static int vfio_device_dma_logging_report(VFIODevice
*vbasedev, hwaddr iova,
     report->iova = iova;
     report->length = size;
     report->page_size = qemu_real_host_page_size();
-    report->bitmap = (__aligned_u64)bitmap;
+    report->bitmap = (__u64)(uintptr_t)bitmap;

     feature->argsz = sizeof(buf);
     feature->flags =
--
2.17.2

[-- Attachment #2: 0014-vfio-common-Fix-32-bit-builds.patch --]
[-- Type: text/plain, Size: 2261 bytes --]

From bbf2c3bbb9c9e97f12dfe49f85dac8cc1f0c5d97 Mon Sep 17 00:00:00 2001
From: Joao Martins <joao.m.martins@oracle.com>
Date: Sun, 5 Mar 2023 18:12:29 -0500
Subject: [PATCH v3 14/13] vfio/common: Fix 32-bit builds

Signed-off-by: Joao Martins <joao.m.martins@oracle.com>
---
 hw/vfio/common.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/hw/vfio/common.c b/hw/vfio/common.c
index 9b909f856722..eecff5bb16c6 100644
--- a/hw/vfio/common.c
+++ b/hw/vfio/common.c
@@ -1554,7 +1554,7 @@ vfio_device_feature_dma_logging_start_create(VFIOContainer *container)
         return NULL;
     }
 
-    control->ranges = (__aligned_u64)ranges;
+    control->ranges = (__u64)(uintptr_t)ranges;
     if (tracking->max32) {
         ranges->iova = tracking->min32;
         ranges->length = (tracking->max32 - tracking->min32) + 1;
@@ -1578,7 +1578,7 @@ static void vfio_device_feature_dma_logging_start_destroy(
     struct vfio_device_feature_dma_logging_control *control =
         (struct vfio_device_feature_dma_logging_control *)feature->data;
     struct vfio_device_feature_dma_logging_range *ranges =
-        (struct vfio_device_feature_dma_logging_range *)control->ranges;
+        (struct vfio_device_feature_dma_logging_range *)(uintptr_t) control->ranges;
 
     g_free(ranges);
     g_free(feature);
@@ -1646,7 +1646,7 @@ static int vfio_device_dma_logging_report(VFIODevice *vbasedev, hwaddr iova,
 {
     uint64_t buf[DIV_ROUND_UP(sizeof(struct vfio_device_feature) +
                         sizeof(struct vfio_device_feature_dma_logging_report),
-                        sizeof(__aligned_u64))] = {};
+                        sizeof(__u64))] = {};
     struct vfio_device_feature *feature = (struct vfio_device_feature *)buf;
     struct vfio_device_feature_dma_logging_report *report =
         (struct vfio_device_feature_dma_logging_report *)feature->data;
@@ -1654,7 +1654,7 @@ static int vfio_device_dma_logging_report(VFIODevice *vbasedev, hwaddr iova,
     report->iova = iova;
     report->length = size;
     report->page_size = qemu_real_host_page_size();
-    report->bitmap = (__aligned_u64)bitmap;
+    report->bitmap = (__u64)(uintptr_t)bitmap;
 
     feature->argsz = sizeof(buf);
     feature->flags =
-- 
2.17.2