Re: Linux Xen Balloon Driver Improvement (Draft 2)

[David Vrabel <david.vrabel@citrix.com>]

On 27/10/14 12:33, Wei Liu wrote:
> 
> Changes in this version:
> 
> 1. Style, grammar and typo fixes.
> 2. Make this document Linux centric.
> 3. Add a new section for NUMA-aware ballooning.

You've not included the required changes to the toolstack and
autoballoon driver to always use 2M multiples when creating VMs and
setting targets.

> ## Introduction
> 
> This document describe a design to improve Xen balloon driver in Linux.

"Linux balloon driver for Xen guests"?

> ## Goal of improvement
> 
> The balloon driver makes use of as many huge pages as possible,
> defragmenting guest address space. Contiguous guest address space
> permits huge page ballooning which helps prevent host address space
> fragmentation.
> 
> This should be achieved without any particular hypervisor side
> feature.

I really think you need to be taking whole-system view and not focusing
on just the guest balloon driver.

> ### Make use of balloon page compaction
> 
> The core of migration callback is XENMEM\_exchange hypercall. This
> makes sure that inflation of old page and deflation of new page is
> done atomically, so even if a domain is beyond its memory target and
> the target is being enforced, it can still compact memory.

Having looked at what XENMEM_exchange actually does, I can't see how
you're using it to give this behaviour.

IMO, XEMMEM_exchange should probably be renamed XENMEM_repopulate or
something.

> ### Periodically exchange normal size pages with huge pages
> 
> Worker thread wakes up periodically to check if there are enough pages
> in normal size page queue to coalesce into a huge page. If so, it will
> try to exchange that huge page into a number of normal size pages with
> XENMEM\_exchange hypercall.

I don't see what this is supposed to achieve.  This is going to take a
(potentially) non-fragmented superpage and fragment it.

Your set of 512 4k ballooned pages needs to be ordered, contiguous and
superpage aligned, for this to be any use.

> ## Relationship with NUMA-aware ballooning
> 
> Another orthogonal improvement to Linux balloon driver is NUMA-aware
> ballooning.
> 
> The use of balloon page compaction will not interfere with NUMA-ware
> ballooning because balloon compaction, which is part of Linux's memory
> subsystem, is already NUMA-aware.
> 
> All the changes proposed in this design can be made NUMA-aware
> provided virtual NUMA topology information is in place.

How?

David