Re: [Qemu-devel] [RFC 00/13] Live memory snapshot based on userfaultfd

[Christian Pinto <c.pinto@virtualopensystems.com>]

Thanks a lot Hailiang


On 28/02/2017 02:48, Hailiang Zhang wrote:
> Hi,
>
> On 2017/2/27 23:37, Christian Pinto wrote:
>> Hello Hailiang,
>>
>> are there any updates on this patch series? Are you planning to release
>> a new version?
>>
>
> No, userfaultfd still does not support write-protect for KVM.
> You can see the newest discussion about it here:
> https://lists.gnu.org/archive/html/qemu-devel/2016-12/msg01127.html
>

Yes, I have read that part of the discussion and quickly managed to 
reproduce the "Bad address" on ARMv8.

>> You say there are some issues with the current snapshot-v2 version,
>> which issues were you referring to? On my side the only problem I have
>> seen was that the live snapshot was not working on ARMv8, but I have
>> fixed that and managed to successfully snapshot and restore a QEMU ARMv8
>> tcg machine on an ARMv8 host. I will gladly contribute with these fixes
>> once you will release a new version of the patches.
>>
>
> Yes, for current implementing of live snapshot, it supports tcg,
> but does not support kvm mode, the reason i have mentioned above,
> if you try to implement it, i think you need to start from userfaultfd
> supporting KVM. There is scenario for it, But I'm blocked by other things
> these days. I'm glad to discuss with you if you planed to do it.
>

I will have a deeper look at why userfault is not yet working with KVM 
and get back on this thread for feedback/suggestions.

Thanks,
Christian


> Thanks.
> Hailiang
>
>>
>> Thanks a lot,
>>
>> Christian
>>
>> On 20/08/2016 08:31, Hailiang Zhang wrote:
>>> Hi,
>>>
>>> I updated this series, but didn't post it, because there are some
>>> problems while i tested the snapshot function.
>>> I didn't know if it is the userfaultfd issue or not.
>>> I don't have time to investigate it this month. I have put them in 
>>> github
>>> https://github.com/coloft/qemu/tree/snapshot-v2
>>>
>>> Anyone who want to test and modify it are welcomed!
>>>
>>> Besides, will you join the linuxcon or KVM forum in Canada ?
>>> I wish to see you there if you join the conference ;)
>>>
>>> Thanks,
>>> Hailiang
>>>
>>>
>>>
>>> On 2016/8/18 23:56, Andrea Arcangeli wrote:
>>>> Hello everyone,
>>>>
>>>> I've an aa.git tree uptodate on the master & userfault branch (master
>>>> includes other pending VM stuff, userfault branch only contains
>>>> userfault enhancements):
>>>>
>>>> https://git.kernel.org/cgit/linux/kernel/git/andrea/aa.git/log/?h=userfault 
>>>>
>>>>
>>>>
>>>> I didn't have time to test KVM live memory snapshot on it yet as I'm
>>>> still working to improve it. Did anybody test it? However I'd be happy
>>>> to take any bugreports and quickly solve anything that isn't working
>>>> right with the shadow MMU.
>>>>
>>>> I got positive report already for another usage of the uffd WP 
>>>> support:
>>>>
>>>> https://medium.com/@MartinCracauer/generational-garbage-collection-write-barriers-write-protection-and-userfaultfd-2-8b0e796b8f7f 
>>>>
>>>>
>>>>
>>>> The last few things I'm working on to finish the WP support are:
>>>>
>>>> 1) pte_swp_mkuffd_wp equivalent of pte_swp_mksoft_dirty to mark in a
>>>>      vma->vm_flags with VM_UFFD_WP set, which swap entries were
>>>>      generated while the pte was wrprotected.
>>>>
>>>> 2) to avoid all false positives the equivalent of pte_mksoft_dirty is
>>>>      needed too... and that requires spare software bits on the pte
>>>>      which are available on x86. I considered also taking over the
>>>>      soft_dirty bit but then you couldn't do checkpoint restore of a
>>>>      JIT/to-native compiler that uses uffd WP support so it wasn't
>>>>      ideal. Perhaps it would be ok as an incremental patch to make the
>>>>      two options mutually exclusive to defer the arch changes that
>>>>      pte_mkuffd_wp would require for later.
>>>>
>>>> 3) prevent UFFDIO_ZEROPAGE if registering WP|MISSING or trigger a
>>>>      cow in userfaultfd_writeprotect.
>>>>
>>>> 4) WP selftest
>>>>
>>>> In theory things should work ok already if the userland code is
>>>> tolerant against false positives through swap and after fork() and
>>>> KSM. For an usage like snapshotting false positives shouldn't be an
>>>> issue (it'll just run slower if you swap in the worst case), and point
>>>> 3) above also isn't an issue because it's going to register into uffd
>>>> with WP only.
>>>>
>>>> The current status includes:
>>>>
>>>> 1) WP support for anon (with false positives.. work in progress)
>>>>
>>>> 2) MISSING support for tmpfs and hugetlbfs
>>>>
>>>> 3) non cooperative support
>>>>
>>>> Thanks,
>>>> Andrea
>>>>
>>>> .
>>>>
>>>
>>
>>
>> .
>>
>