Re: [PATCH v3 00/13] vfio/migration: Device dirty page tracking

["Cédric Le Goater" <clg@redhat.com>]

On 3/6/23 10:45, Joao Martins wrote:
> On 06/03/2023 02:19, Alex Williamson wrote:
>> On Sun, 5 Mar 2023 23:33:35 +0000
>> Joao Martins <joao.m.martins@oracle.com> wrote:
>>
>>> On 05/03/2023 20:57, Alex Williamson wrote:
>>>> On Sat,  4 Mar 2023 01:43:30 +0000
>>>> Joao Martins <joao.m.martins@oracle.com> wrote:
>>>>    
>>>>> Hey,
>>>>>
>>>>> Presented herewith a series based on the basic VFIO migration protocol v2
>>>>> implementation [1].
>>>>>
>>>>> It is split from its parent series[5] to solely focus on device dirty
>>>>> page tracking. Device dirty page tracking allows the VFIO device to
>>>>> record its DMAs and report them back when needed. This is part of VFIO
>>>>> migration and is used during pre-copy phase of migration to track the
>>>>> RAM pages that the device has written to and mark those pages dirty, so
>>>>> they can later be re-sent to target.
>>>>>
>>>>> Device dirty page tracking uses the DMA logging uAPI to discover device
>>>>> capabilities, to start and stop tracking, and to get dirty page bitmap
>>>>> report. Extra details and uAPI definition can be found here [3].
>>>>>
>>>>> Device dirty page tracking operates in VFIOContainer scope. I.e., When
>>>>> dirty tracking is started, stopped or dirty page report is queried, all
>>>>> devices within a VFIOContainer are iterated and for each of them device
>>>>> dirty page tracking is started, stopped or dirty page report is queried,
>>>>> respectively.
>>>>>
>>>>> Device dirty page tracking is used only if all devices within a
>>>>> VFIOContainer support it. Otherwise, VFIO IOMMU dirty page tracking is
>>>>> used, and if that is not supported as well, memory is perpetually marked
>>>>> dirty by QEMU. Note that since VFIO IOMMU dirty page tracking has no HW
>>>>> support, the last two usually have the same effect of perpetually
>>>>> marking all pages dirty.
>>>>>
>>>>> Normally, when asked to start dirty tracking, all the currently DMA
>>>>> mapped ranges are tracked by device dirty page tracking. If using a
>>>>> vIOMMU we block live migration. It's temporary and a separate series is
>>>>> going to add support for it. Thus this series focus on getting the
>>>>> ground work first.
>>>>>
>>>>> The series is organized as follows:
>>>>>
>>>>> - Patches 1-7: Fix bugs and do some preparatory work required prior to
>>>>>    adding device dirty page tracking.
>>>>> - Patches 8-10: Implement device dirty page tracking.
>>>>> - Patch 11: Blocks live migration with vIOMMU.
>>>>> - Patches 12-13 enable device dirty page tracking and document it.
>>>>>
>>>>> Comments, improvements as usual appreciated.
>>>>
>>>> Still some CI failures:
>>>>
>>>> https://gitlab.com/alex.williamson/qemu/-/pipelines/796657474
>>>>
>>>> The docker failures are normal, afaict the rest are not.  Thanks,
>>>>    
>>>
>>> Ugh, sorry
>>>
>>> The patch below scissors mark (and also attached as a file) fixes those build
>>> issues. I managed to reproduce on i386 target builds, and these changes fix my
>>> 32-bit build.
>>>
>>> I don't have a working Gitlab setup[*] though to trigger the CI to enable to
>>> wealth of targets it build-tests. If you could kindly test the patch attached in
>>> a new pipeline (applied on top of the branch you just build) below to understand
>>> if the CI gets happy. I will include these changes in the right patches (patch 8
>>> and 10) for the v4 spin.
>>
>> Looks like this passes:
>>
>> https://gitlab.com/alex.williamson/qemu/-/pipelines/796750136
>>
> Great, I've staged this fixes in patches 8&10!
> 
> I have a sliver of hope that we might still make it by soft freeze (tomorrow?).
> If you think it can still make it, should the rest of the series is good, then I
> can follow up v4 today/tomorrow. Thoughts?

I would say, wait and see if a v4 is needed first. These changes are
relatively easy to fold in.

C.



> 
> 	Joao
>