[Qemu-devel] balloon driver

[Qemu-devel] balloon driver

[Eric L]

Hi all,

I've been playing around with QEMU the past few days and have been
quite impressed.  One thing I wondered about: it seems that most of
the other virtualization schemes have some sort of balloon driver to
reclaim unused (cached) guest memory.  (VMWare, Xen and I think user
mode linux has something too).  It seems like a pretty good idea to
me, but from my searching, it would appear that QEMU does not have
anything similar.  Is this correct?  If so, is there a reason why?


- E

Re: [Qemu-devel] balloon driver

[Paul Brook]

On Wednesday 05 July 2006 03:58, Eric L wrote:
> I've been playing around with QEMU the past few days and have been
> quite impressed.  One thing I wondered about: it seems that most of
> the other virtualization schemes have some sort of balloon driver to
> reclaim unused (cached) guest memory.  (VMWare, Xen and I think user
> mode linux has something too).  It seems like a pretty good idea to
> me, but from my searching, it would appear that QEMU does not have
> anything similar.  Is this correct?  If so, is there a reason why?

Partly because qemu is just a normal user application. It can be swapped out 
by the host OS just like any other process. Adding a few Gb of extra swap and 
letting the host OS figure it out should get you 90% of the benefit.

Paul

Re: [Qemu-devel] balloon driver

[Eric L]

On 7/5/06, Paul Brook <paul@codesourcery.com> wrote:
> On Wednesday 05 July 2006 03:58, Eric L wrote:
> > I've been playing around with QEMU the past few days and have been
> > quite impressed.  One thing I wondered about: it seems that most of
> > the other virtualization schemes have some sort of balloon driver to
> > reclaim unused (cached) guest memory.  (VMWare, Xen and I think user
> > mode linux has something too).  It seems like a pretty good idea to
> > me, but from my searching, it would appear that QEMU does not have
> > anything similar.  Is this correct?  If so, is there a reason why?
>
> Partly because qemu is just a normal user application. It can be swapped out
> by the host OS just like any other process. Adding a few Gb of extra swap and
> letting the host OS figure it out should get you 90% of the benefit.
>
> Paul
>

It seems the point of the balloon driver is to avoid forcing the host
to swap.  For example, suppose I start a new guest OS.  I check the
memory usage on the host and everything looks pretty good, maybe 30MB
used.  Then suppose I run a recursive grep command in a Linux source
tree on the guest.  The host memory usage will climb to the maximum
allotted memory as the guest OS fills its page cache with pages of
kernel source.  Now, I go back to the host and decide I want to run
something a little memory intensive.  The host has to swap and
dutifully copies those pages of kernel source to swap.  Much better
would be if I could just chuck those pages and give them back to the
host, no swapping at all.

Even if the guest has to swap, the reasoning is that the guest is in a
much better position to figure out what to swap than if the host were
forced to.

It is a rather crude approach and I'm not sure how much practical
benefit there is, but I'll probably go ahead and code it up (at least
for a Linux host) if only for myself as it looks pretty simple.
(Linux 2.6.16 added the ability to punch holes in tmpfs files so all
the hard work should be done).  I just wondered if there was anything
I was missing or if anyone had considered it before.


- E

Re: [Qemu-devel] balloon driver

[Mark Williamson]

> It seems the point of the balloon driver is to avoid forcing the host
> to swap.  For example, suppose I start a new guest OS.  I check the
> memory usage on the host and everything looks pretty good, maybe 30MB
> used.  Then suppose I run a recursive grep command in a Linux source
> tree on the guest.  The host memory usage will climb to the maximum
> allotted memory as the guest OS fills its page cache with pages of
> kernel source.  Now, I go back to the host and decide I want to run
> something a little memory intensive.  The host has to swap and
> dutifully copies those pages of kernel source to swap.  Much better
> would be if I could just chuck those pages and give them back to the
> host, no swapping at all.
>
> Even if the guest has to swap, the reasoning is that the guest is in a
> much better position to figure out what to swap than if the host were
> forced to.

Worse, the guest might decide to swap out a page that's already swapped in by 
the host, forcing it to be read in again only to be immediately written out 
to disk by the guest :-(

> It is a rather crude approach and I'm not sure how much practical
> benefit there is, but I'll probably go ahead and code it up (at least
> for a Linux host) if only for myself as it looks pretty simple.
> (Linux 2.6.16 added the ability to punch holes in tmpfs files so all
> the hard work should be done).  I just wondered if there was anything
> I was missing or if anyone had considered it before.

Xen has a balloon driver - you might like to take a look at that for starters, 
and maybe borrow some code from it!  There's been occasional talk to the 
effect that a number of projects might as well share functionality like 
ballooning since a number of systems require it.

Cheers,
Mark

>
> - E
>
>
> _______________________________________________
> Qemu-devel mailing list
> Qemu-devel@nongnu.org
> http://lists.nongnu.org/mailman/listinfo/qemu-devel

-- 
Dave: Just a question. What use is a unicyle with no seat?  And no pedals!
Mark: To answer a question with a question: What use is a skateboard?
Dave: Skateboards have wheels.
Mark: My wheel has a wheel!

Re: [Qemu-devel] balloon driver

[Jamie Lokier]

Mark Williamson wrote:
> > Even if the guest has to swap, the reasoning is that the guest is in a
> > much better position to figure out what to swap than if the host were
> > forced to.
> 
> Worse, the guest might decide to swap out a page that's already
> swapped in by the host, forcing it to be read in again only to be
> immediately written out to disk by the guest :-(

...unless the guest's disk I/O is with simulated DMA or recognisable
block-copy instruction sequences, and doesn't look at the data.  In
that case the emulator can, in principle, keep track of where pages
are copied around without being examined, and avoid actually swapping
them in.

Probably not worth the complexity.

-- Jamie

Re: [Qemu-devel] balloon driver

[Mark Williamson]

> > Worse, the guest might decide to swap out a page that's already
> > swapped in by the host, forcing it to be read in again only to be
> > immediately written out to disk by the guest :-(
>
> ...unless the guest's disk I/O is with simulated DMA or recognisable
> block-copy instruction sequences, and doesn't look at the data.  In
> that case the emulator can, in principle, keep track of where pages
> are copied around without being examined, and avoid actually swapping
> them in.
>
> Probably not worth the complexity.

Another solution, taken by the Disco VMM was to provide a special virtual disk 
device for the swap device which fakes out swapping behaviour - accesses to 
that device are always going to be swaps, so you can guarantee to do the 
right thing without having to trace and interpret behaviour.

That said, I like your idea of not committing IOs for DMA-ed pages than have 
not been inspected, but one would have to be careful to make sure that true 
page-size DMA writes of data do still hit the disk in a timely fashion.

Cheers,
Mark

-- 
Dave: Just a question. What use is a unicyle with no seat?  And no pedals!
Mark: To answer a question with a question: What use is a skateboard?
Dave: Skateboards have wheels.
Mark: My wheel has a wheel!