Re: Xen balloon driver improvement (version 1)

["Roger Pau Monné" <roger.pau@citrix.com>]

El 23/10/14 a les 17.23, Wei Liu ha escrit:
> On Thu, Oct 23, 2014 at 04:30:24PM +0200, Roger Pau Monné wrote:
>> El 22/10/14 a les 18.29, Wei Liu ha escrit:
>>> Hi all
>>>
>>> This is my initial design to improve Xen balloon driver.
>>>
>>> PDF version with graphs can be found at
>>>
>>> http://xenbits.xen.org/people/liuw/xen-balloon-driver-improvement.pdf
>>>
>>> % Xen Balloon Driver Improvement
>>> % Wei Liu <<wei.liu2@citrix.com>>
>>>
>>> -------------------------------------------
>>> Version     Date         Changes
>>> -------     ----         ------------------
>>>   1         22/10/2014   Initial version.
>>> -------------------------------------------
>>>
>>> ## Motives
>>>
>>> 1. Balloon pages fragments guest physical address space.
>>> 1. Balloon compaction infrastructure can migrate ballooned pages from
>>>    start of zone to end of zone, hence creating contiguous guest physical
>>>    address space.
>>> 1. Having contiguous guest physical address enables some options to
>>>    improve performance.
>>>
>>> ## Goal of improvement
>>>
>>> Balloon driver makes use of as many huge pages as possible,
>>> defragmenting both guest address space and Xen pages. This should be
>>> achieved without any particular hypervisor side feature.
>>>
>>> ## Design and implementation
>>>
>>> When balloon driver is asked to increase / decrease reservation, it
>>> will always start with huge page. However, due to resource
>>> availability in both hypervisor and guest, it's not always possible to
>>> get hold of a huge page. In that case the driver will fall back to use
>>> normal size page. Balloon driver later will try to coalesce small size
>>> pages into huge page. As time goes by, both Xen and guest should use
>>> more and more huge pages.
>>
>> All this looks quite complicated IMHO, it's adding a lot of logic to the
>> balloon driver. Can't you just ask the memory subsystem to allocate a
>> page (or pages) from a specific physical range, and force it to page
>> out/move what's there at allocation time?
>>
>> For example I know FreeBSD has contigmalloc(9)[1] which I think could be
>> used to achieve this. You could start asking for pages starting at
>> maxpfn and go down from there, keeping fragmentation at a minimum.
>>
>> [1]
>> https://www.freebsd.org/cgi/man.cgi?query=contigmalloc&apropos=0&sektion=0&manpath=FreeBSD+10.1-RELEASE&arch=default&format=html
>>
> 
> Good point. Just that Linux doesn't have a counterpart, not that I know
> of. Memblock infrastructure looks similar but it's supposed to be used
> when initialising kernel.
> 
> Even if Linux has similar API, it's still less desirable because to
> satisfy a contiguous PA allocation, the system needs to be relative
> quiet (if NO_WAIT / ATOMIC is set), or the API needs to sleep for
> indefinite period (wait for memory subsystem to squeeze out pages).

There's no restriction on the time it might take for a guest to balloon
out. IMHO I would rather add a new interface to the Linux VM subsystem
that tries to accomplish this rather than adding a bunch of logic
specific to the balloon driver.

In general you should be able to reclaim memory quite fast (by either
moving it to another region or swapping it to disk). In case of finding
a page that's wired I would just leave it as is, since I guess this
would not be quite common, and maybe retry after a certain period.

Roger.