[Qemu-devel] [BUG] Balloon malfunctions with memory hotplug

[Luiz Capitulino <lcapitulino@redhat.com>]

Hello,

Reproducer:

1. Start QEMU with balloon and memory hotplug support:

# qemu [...] -m 1G,slots=2,maxmem=2G -balloon virtio

2. Check balloon size:

(qemu) info balloon
balloon: actual=1024
(qemu) 

3. Hotplug some memory:

(qemu) object_add memory-backend-ram,id=mem1,size=1G
(qemu) device_add pc-dimm,id=dimm1,memdev=mem1

4. This is step is _not_ needed to reproduce the problem,
   but you may need to online memory manually on Linux so
   that it becomes available in the guest

5. Check balloon size again:

(qemu) info balloon
balloon: actual=1024
(qemu) 

BUG: The guest now has 2GB of memory, but the balloon thinks
     the guest has 1GB

One may think that the problem is that the balloon driver is
ignoring hotplugged memory. This is not what's happening. If
you do balloon your guest, there's nothing stopping the
balloon driver in the guest from ballooning hotplugged memory.

The problem is that the balloon device in QEMU needs to know
the current amount of memory available to the guest.

Before memory hotplug this information was easy to obtain: the
current amount of memory available to the guest is the memory the
guest was booted with. This value is stored in the ram_size global
variable in QEMU and this is what the balloon device emulation
code uses today. However, when memory is hotplugged ram_size is
_not_ updated and the balloon device breaks.

I see two possible solutions for this problem:

1. In addition to reading ram_size, the balloon device in QEMU
   could scan pc-dimm devices to account for hotplugged memory.

   This solution was already implemented by zhanghailiang:

    http://lists.gnu.org/archive/html/qemu-devel/2014-11/msg02362.html

   It works, except that on Linux memory hotplug is a two-step
   procedure: first memory is inserted then it has to be onlined
   from user-space. So, if memory is inserted but not onlined
   this solution gives the opposite problem: the balloon device
   will report a larger memory amount than the guest actually has.

   Can we live with that? I guess not, but I'm open for discussion.

   If QEMU could be notified when Linux makes memory online, then
   the problem would be gone. But I guess this can't be done.

2. Modify the balloon driver in the guest to inform the balloon
   device on the host about the current memory available to the
   guest. This way, whenever the balloon device in QEMU needs
   to know the current amount of memory in the guest, it asks
   the guest. This drops any usage of ram_size in the balloon
   device

   I'm not completely sure this is feasible though. For example,
   what happens if the guest reports a memory amount to QEMU and
   right after this more memory is plugged?

   Besides, this solution is more complex than solution 1 and
   won't address older guests.

Another important detail is that, I *suspect* that a very similar
bug already exists with 32-bit guests even without memory
hotplug: what happens if you assign 6GB to a 32-bit without PAE
support? I think the same problem we're seeing with memory
hotplug will happen and solution 1 won't fix this, although
no one seems to care about 32-bit guests...