Re: Xen balloon driver improvement (version 1)

[Ian Campbell <Ian.Campbell@citrix.com>]

On Wed, 2014-10-22 at 17:29 +0100, Wei Liu wrote:

> For instance, balloon driver can maintain three queues:
> 
> 1. queue for 2 MB pages
> 1. queue for 4 KB pages (delegated to core balloon driver)
> 1. queue for pages used to mapped pages from other domain

I think I'd describe this last one as "pages used to provide empty
address ranges to drivers" or something like that. Yes, they will
probably be used for mappings, but I don't think that is the only use of
the alloc_xenballooned_pages interface.

On that subject, how do you handle alloc_xenballooned_pages calls of
non-2M alignment? Would it be best to do a 2M balloon and queue the rest
for use on future similar allocations?

If so then I'm wondering if it might make sense to keep the spare 4K
pages from doing this on a separate queue to the normal 4K queue, in
order to keep these sorts pages isolated into 2M regions -- because I
expect that they cannot be compacted without cooperation with the driver
which allocated them (which I expect won't even be possible in many
cases).

> These flowcharts assume normal page size is 4K and huge page size is
> 2M.  They show how two queues are maintained.
> 
> ![Increase Reservation](increase-reservation.png)

There's a few implicit "requeue on failure" arcs missing on some of
these, I think adding them would make the picture hard to follow, but
perhaps a footnote?
 
On both here and decrease there are attempts to allocate (from either
the queue or the kernel, depending) 2M which could fail. I wonder if it
is worth inserting "kick THP and give it a chance"? It's a question of
tradeoffs in the latency of a ballooning operation vs the efficiency
with which we can use 2M allocations.

> ![Decrease Reservation](decrease-reservation.png)
> 
> ![Exchange Pages](exchange-pages.png)
>