Re: [PULL 05/10] Add Hyper-V Dynamic Memory Protocol driver (hv-balloon) base

["Philippe Mathieu-Daudé" <philmd@linaro.org>]

On 11/3/25 16:11, Maciej S. Szmigiero wrote:
> On 11.03.2025 16:00, Philippe Mathieu-Daudé wrote:
>> Hi Maciej,
>>
>> On 6/11/23 15:20, Maciej S. Szmigiero wrote:
>>> From: "Maciej S. Szmigiero" <maciej.szmigiero@oracle.com>
>>>
>>> This driver is like virtio-balloon on steroids: it allows both 
>>> changing the
>>> guest memory allocation via ballooning and (in the next patch) inserting
>>> pieces of extra RAM into it on demand from a provided memory backend.
>>>
>>> The actual resizing is done via ballooning interface (for example, via
>>> the "balloon" HMP command).
>>> This includes resizing the guest past its boot size - that is, hot- 
>>> adding
>>> additional memory in granularity limited only by the guest alignment
>>> requirements, as provided by the next patch.
>>>
>>> In contrast with ACPI DIMM hotplug where one can only request to 
>>> unplug a
>>> whole DIMM stick this driver allows removing memory from guest in single
>>> page (4k) units via ballooning.
>>>
>>> After a VM reboot the guest is back to its original (boot) size.
>>>
>>> In the future, the guest boot memory size might be changed on reboot
>>> instead, taking into account the effective size that VM had before that
>>> reboot (much like Hyper-V does).
>>>
>>> For performance reasons, the guest-released memory is tracked in a few
>>> range trees, as a series of (start, count) ranges.
>>> Each time a new page range is inserted into such tree its neighbors are
>>> checked as candidates for possible merging with it.
>>>
>>> Besides performance reasons, the Dynamic Memory protocol itself uses 
>>> page
>>> ranges as the data structure in its messages, so relevant pages need 
>>> to be
>>> merged into such ranges anyway.
>>>
>>> One has to be careful when tracking the guest-released pages, since the
>>> guest can maliciously report returning pages outside its current address
>>> space, which later clash with the address range of newly added memory.
>>> Similarly, the guest can report freeing the same page twice.
>>>
>>> The above design results in much better ballooning performance than when
>>> using virtio-balloon with the same guest: 230 GB / minute with this 
>>> driver
>>> versus 70 GB / minute with virtio-balloon.
>>>
>>> During a ballooning operation most of time is spent waiting for the 
>>> guest
>>> to come up with newly freed page ranges, processing the received 
>>> ranges on
>>> the host side (in QEMU and KVM) is nearly instantaneous.
>>>
>>> The unballoon operation is also pretty much instantaneous:
>>> thanks to the merging of the ballooned out page ranges 200 GB of 
>>> memory can
>>> be returned to the guest in about 1 second.
>>> With virtio-balloon this operation takes about 2.5 minutes.
>>>
>>> These tests were done against a Windows Server 2019 guest running on a
>>> Xeon E5-2699, after dirtying the whole memory inside guest before each
>>> balloon operation.
>>>
>>> Using a range tree instead of a bitmap to track the removed memory also
>>> means that the solution scales well with the guest size: even a 1 TB 
>>> range
>>> takes just a few bytes of such metadata.
>>>
>>> Since the required GTree operations aren't present in every Glib version
>>> a check for them was added to the meson build script, together with new
>>> "--enable-hv-balloon" and "--disable-hv-balloon" configure arguments.
>>> If these GTree operations are missing in the system's Glib version this
>>> driver will be skipped during QEMU build.
>>>
>>> An optional "status-report=on" device parameter requests memory status
>>> events from the guest (typically sent every second), which allow the 
>>> host
>>> to learn both the guest memory available and the guest memory in use
>>> counts.
>>>
>>> Following commits will add support for their external emission as
>>> "HV_BALLOON_STATUS_REPORT" QMP events.
>>>
>>> The driver is named hv-balloon since the Linux kernel client driver for
>>> the Dynamic Memory Protocol is named as such and to follow the naming
>>> pattern established by the virtio-balloon driver.
>>> The whole protocol runs over Hyper-V VMBus.
>>>
>>> The driver was tested against Windows Server 2012 R2, Windows Server 
>>> 2016
>>> and Windows Server 2019 guests and obeys the guest alignment 
>>> requirements
>>> reported to the host via DM_CAPABILITIES_REPORT message.
>>>
>>> Acked-by: David Hildenbrand <david@redhat.com>
>>> Signed-off-by: Maciej S. Szmigiero <maciej.szmigiero@oracle.com>
>>> ---
>>>   Kconfig.host                           |    3 +
>>>   hw/hyperv/Kconfig                      |   10 +
>>>   hw/hyperv/hv-balloon-internal.h        |   33 +
>>>   hw/hyperv/hv-balloon-page_range_tree.c |  228 +++++
>>>   hw/hyperv/hv-balloon-page_range_tree.h |  118 +++
>>>   hw/hyperv/hv-balloon.c                 | 1160 ++++++++++++++++++++++++
>>>   hw/hyperv/meson.build                  |    1 +
>>>   hw/hyperv/trace-events                 |   13 +
>>>   include/hw/hyperv/hv-balloon.h         |   18 +
>>>   meson.build                            |   28 +-
>>>   meson_options.txt                      |    2 +
>>>   scripts/meson-buildoptions.sh          |    3 +
>>>   12 files changed, 1616 insertions(+), 1 deletion(-)
>>>   create mode 100644 hw/hyperv/hv-balloon-internal.h
>>>   create mode 100644 hw/hyperv/hv-balloon-page_range_tree.c
>>>   create mode 100644 hw/hyperv/hv-balloon-page_range_tree.h
>>>   create mode 100644 hw/hyperv/hv-balloon.c
>>>   create mode 100644 include/hw/hyperv/hv-balloon.h
>>>
>>> diff --git a/Kconfig.host b/Kconfig.host
>>> index d763d892693c..2ee71578f38f 100644
>>> --- a/Kconfig.host
>>> +++ b/Kconfig.host
>>> @@ -46,3 +46,6 @@ config FUZZ
>>>   config VFIO_USER_SERVER_ALLOWED
>>>       bool
>>>       imply VFIO_USER_SERVER
>>> +
>>> +config HV_BALLOON_POSSIBLE
>>> +    bool
>>> diff --git a/hw/hyperv/Kconfig b/hw/hyperv/Kconfig
>>> index fcf65903bd05..41dd827c841b 100644
>>> --- a/hw/hyperv/Kconfig
>>> +++ b/hw/hyperv/Kconfig
>>> @@ -16,3 +16,13 @@ config SYNDBG
>>>       bool
>>>       default y
>>>       depends on VMBUS
>>> +
>>> +config HV_BALLOON_SUPPORTED
>>> +    bool
>>> +
>>> +config HV_BALLOON
>>> +    bool
>>> +    default y
>>> +    depends on VMBUS
>>> +    depends on HV_BALLOON_POSSIBLE
>>
>> Where is HV_BALLOON_POSSIBLE set?
> 
> In meson.build:
>> host_kconfig = \
> (..)
>>  (hv_balloon ? ['CONFIG_HV_BALLOON_POSSIBLE=y'] : []) + \
> 
> Then meson passes this as a command-line parameter
> to scripts/minikconf.py, together with other similar
> config values in host_kconfig.
> 
> See build/meson-logs/meson-log.txt for scripts/minikconf.py
> complete command line.

Right, thank you!