Re: [PATCH v5 00/11] virtio-mem: vfio support

[David Hildenbrand <david@redhat.com>]

On 16.02.21 19:33, Alex Williamson wrote:
> On Mon, 15 Feb 2021 15:03:43 +0100
> David Hildenbrand <david@redhat.com> wrote:
> 
>> On 08.02.21 09:28, David Hildenbrand wrote:
>>> On 27.01.21 13:45, Michael S. Tsirkin wrote:
>>>> On Thu, Jan 21, 2021 at 12:05:29PM +0100, David Hildenbrand wrote:
>>>>> A virtio-mem device manages a memory region in guest physical address
>>>>> space, represented as a single (currently large) memory region in QEMU,
>>>>> mapped into system memory address space. Before the guest is allowed to use
>>>>> memory blocks, it must coordinate with the hypervisor (plug blocks). After
>>>>> a reboot, all memory is usually unplugged - when the guest comes up, it
>>>>> detects the virtio-mem device and selects memory blocks to plug (based on
>>>>> resize requests from the hypervisor).
>>>>>
>>>>> Memory hot(un)plug consists of (un)plugging memory blocks via a virtio-mem
>>>>> device (triggered by the guest). When unplugging blocks, we discard the
>>>>> memory - similar to memory balloon inflation. In contrast to memory
>>>>> ballooning, we always know which memory blocks a guest may actually use -
>>>>> especially during a reboot, after a crash, or after kexec (and during
>>>>> hibernation as well). Guests agreed to not access unplugged memory again,
>>>>> especially not via DMA.
>>>>>
>>>>> The issue with vfio is, that it cannot deal with random discards - for this
>>>>> reason, virtio-mem and vfio can currently only run mutually exclusive.
>>>>> Especially, vfio would currently map the whole memory region (with possible
>>>>> only little/no plugged blocks), resulting in all pages getting pinned and
>>>>> therefore resulting in a higher memory consumption than expected (turning
>>>>> virtio-mem basically useless in these environments).
>>>>>
>>>>> To make vfio work nicely with virtio-mem, we have to map only the plugged
>>>>> blocks, and map/unmap properly when plugging/unplugging blocks (including
>>>>> discarding of RAM when unplugging). We achieve that by using a new notifier
>>>>> mechanism that communicates changes.
>>>>
>>>> series
>>>>
>>>> Acked-by: Michael S. Tsirkin <mst@redhat.com>
>>>>
>>>> virtio bits
>>>>
>>>> Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
>>>>
>>>> This needs to go through vfio tree I assume.
>>>
>>> Thanks Michael.
>>>
>>> @Alex, what are your suggestions?
>>
>> Gentle ping.
> 
> Sorry for the delay.  It looks to me like patches 1, 8, and 9 are
> Memory API that are still missing an Ack from Paolo.  I'll toss in my
> A-b+R-b for patches 6 and 7.  I don't see that this necessarily needs
> to go in through vfio, I'm more than happy if someone else wants to
> grab it.  Thanks,

Thanks, I assume patch #11 is fine with you as well?

@Paolo, it would be great if I can get your feedback on patch #1. I have 
more stuff coming up that will reuse RamDiscardMgr (i.e., for better 
migration handling and better guest memory dump handling).

-- 
Thanks,

David / dhildenb