[LSF/MM TOPIC] [ATTEND] memory compaction & ballooning

[LSF/MM TOPIC] [ATTEND] memory compaction & ballooning

[Rik van Riel]

KVM guests benefit a lot from being able to use transparent
hugepages in both the host and the guest, cutting the number
of memory accesses to fill a TLB entry almost in half when
using EPT/NPT.

However, currently memory is ballooned in 4kB units, leading
to fragmentation of guest memory and an inability to use 2MB
pages.  One obvious fix is to do memory ballooning in 2MB
increments, however there appear to be several obstacles in
the way of compaction actually creating contiguous 2MB areas
of free memory.

I would like to discuss / brainstorm improvements to
compaction and ways to keep memory allocations better
separated, to be better able to come up with contiguous
2MB areas.


-- 
All rights reversed

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [LSF/MM TOPIC] [ATTEND] memory compaction & ballooning

[Dan Magenheimer]

Re: http://marc.info/?l=linux-mm&m=132732724710038&w=2 
(sorry I couldn't properly thread this)

> I would like to discuss / brainstorm improvements to
> compaction and ways to keep memory allocations better
> separated, to be better able to come up with contiguous
> 2MB areas.

Hi Rik --

I have some thoughts on how in-kernel cleancache/tmem can be
used to help achieve this.  We can talk about it more
in April, but basically:

- IMHO the big issue with superpage ballooning is that you need
  to either maintain a ready-to-use free-list of superpages
  (but keeping this list is far too wasteful of space); or you
  need to do just-in-time compaction of a lot of superpages
  which is time-consuming (periodic "pauses") and subject
  to failure due to fragmentation.
- Under the conditions where you would want to do ballooning,
  most of the wasted space in a superpage free list would
  otherwise be used for clean file-mapped pagecache pages.
  So a large superpage free list would lead to a potentially
  large increase in refaults (and thus disk reads).

So:

- Implement a zcache-like driver that allocates 2MB superpages
  and uses each to store large quantities of cleancache 4K
  pages (compressed or not) AND all associated meta-data,
  such that all 4K pages stored in the superpage can be instantly
  evicted from cleancache simply by removing the superpage
  from a list.
- This list of reclaimable superpages would be primarily used by
  the balloon superpage driver but can also be reclaimed under
  certain low-memory conditions.
- If implemented properly, there is no internal fragmentation
  (in the non-compression case).

Of course this is not free either; it serves a similar
purpose to compaction but amortizes the cost over time.
And you have the choice of compression, which increases
density by increasing CPU cost, or non-compression which
has only page-copy overhead and some small meta-data space
overhead.

Dan

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>