Balloon driver?

Balloon driver?

[Jun Koi]

Hi,

I want to try the balloon driver. Do we have it in any git tree?

The git tree I am having now only has master branch, and nothing else.

(Maybe the balloon driver is not working yet, but I am curious when
see the userspace code in user/)

Thanks,
Jun

-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/

Re: Balloon driver?

[Dor Laor]

>Hi,
>
>I want to try the balloon driver. Do we have it in any git tree?
>
>The git tree I am having now only has master branch, and nothing else.
>
>(Maybe the balloon driver is not working yet, but I am curious when
>see the userspace code in user/)
>
>Thanks,
>Jun

You're welcome to test it, it is under
git://kvm.qumranet.com/home/dor/src/linux-2.6
Checkout the build-trunk branch.

I don't have the time to merge it currently. Note that there are also
some more balloon drivers for other architecture (s390).

The Linux guest driver change the balloon size using a simple char
device.
In our windows drivers we control the balloon from the host using pci
device. Qemu handle/controls the balloon hypercalls for mgmt reasons.

-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/

Re: Balloon driver?

[Jun Koi]

Hi Dor,

On 6/5/07, Dor Laor <dor.laor-atKUWr5tajBWk0Htik3J/w@public.gmane.org> wrote:
> >Hi,
> >
> >I want to try the balloon driver. Do we have it in any git tree?
> >
> >The git tree I am having now only has master branch, and nothing else.
> >
> >(Maybe the balloon driver is not working yet, but I am curious when
> >see the userspace code in user/)
> >
> >Thanks,
> >Jun
>
> You're welcome to test it, it is under
> git://kvm.qumranet.com/home/dor/src/linux-2.6
> Checkout the build-trunk branch.
>
> I don't have the time to merge it currently. Note that there are also
> some more balloon drivers for other architecture (s390).
>
> The Linux guest driver change the balloon size using a simple char
> device.

Now I see the userspace tool again, and it seems it must be used from
inside the guest??

So we cannot balloon up/down a guest from host machine?

I will try to see if it works.

Thanks,
Jun

-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/

Re: Balloon driver?

[Dor Laor]

>> >I want to try the balloon driver. Do we have it in any git tree?
>> >
>> >The git tree I am having now only has master branch, and nothing
else.
>> >
>> >(Maybe the balloon driver is not working yet, but I am curious when
>> >see the userspace code in user/)
>> >
>> >Thanks,
>> >Jun
>>
>> You're welcome to test it, it is under
>> git://kvm.qumranet.com/home/dor/src/linux-2.6
>> Checkout the build-trunk branch.
>>
>> I don't have the time to merge it currently. Note that there are also
>> some more balloon drivers for other architecture (s390).
>>
>> The Linux guest driver change the balloon size using a simple char
>> device.
>
>Now I see the userspace tool again, and it seems it must be used from
>inside the guest??

That's right. That's the Linux guest side implementation.
For windows we use the vmchannel option in qemu (qemu/hw/hypercall)
interface. It is a qemu device from the host side that can be connected
to tcp/unix socket. The guest sees it as a pci device.
Both ends can define any protocol over the channel.

So our windows guest passes balloon inflate/deflate commands/acks.

What's your motivation of ballooning? If it's important I can add the
pci interface + push it upstream.
>
>So we cannot balloon up/down a guest from host machine?
>
>I will try to see if it works.
>
>Thanks,
>Jun



-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/

Re: Balloon driver?

[Jun Koi]

On 6/6/07, Dor Laor <dor.laor-atKUWr5tajBWk0Htik3J/w@public.gmane.org> wrote:
> >> >I want to try the balloon driver. Do we have it in any git tree?
> >> >
> >> >The git tree I am having now only has master branch, and nothing
> else.
> >> >
> >> >(Maybe the balloon driver is not working yet, but I am curious when
> >> >see the userspace code in user/)
> >> >
> >> >Thanks,
> >> >Jun
> >>
> >> You're welcome to test it, it is under
> >> git://kvm.qumranet.com/home/dor/src/linux-2.6
> >> Checkout the build-trunk branch.
> >>
> >> I don't have the time to merge it currently. Note that there are also
> >> some more balloon drivers for other architecture (s390).
> >>
> >> The Linux guest driver change the balloon size using a simple char
> >> device.
> >
> >Now I see the userspace tool again, and it seems it must be used from
> >inside the guest??
>
> That's right. That's the Linux guest side implementation.
> For windows we use the vmchannel option in qemu (qemu/hw/hypercall)
> interface. It is a qemu device from the host side that can be connected
> to tcp/unix socket. The guest sees it as a pci device.
> Both ends can define any protocol over the channel.
>
> So our windows guest passes balloon inflate/deflate commands/acks.
>
> What's your motivation of ballooning? If it's important I can add the
> pci interface + push it upstream.

I looked at the current balloon driver of Linux, and really I dont
like it. Why we have a different solution from Windows'? It is better
we have the same way of implementing it on different OS.

So the PCI interface is definetely better IMO. Please put it online,
so we can test it.

Thanks,
Jun

-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/

Re: Balloon driver?

[Carsten Otte]

Jun Koi wrote:
> So the PCI interface is definetely better IMO. Please put it online,
> so we can test it.
Why would we want a PCI interface to do balooning? All we need is a 
simple interface. It _is_ a mess for platforms that don't have PCI. 
And it is a mess for paravirtulaization - will we create a virtual PCI 
bus just for this?

so long,
Carsten

-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/

Re: Balloon driver?

[Dor Laor]

>Jun Koi wrote:
>> So the PCI interface is definetely better IMO. Please put it online,
>> so we can test it.
>Why would we want a PCI interface to do balooning? All we need is a
>simple interface. It _is_ a mess for platforms that don't have PCI.
>And it is a mess for paravirtulaization - will we create a virtual PCI
>bus just for this?

I agree that pci interface isn't ideal, the advantage is that platforms
that do support pci don't have to implement a virtualized bus, and pci
is PnP device. This was the motivation behind using it, especially for
Windows.

Now that more platforms are joining the KVM wagon, we should define a
common bus. PCI was a overkill anyway - its irq are shared and we don't
have to use its io/mmio areas.

Do you guys have something to start with?


>
>so long,
>Carsten

-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/

Re: Balloon driver?

[Carsten Otte]

Dor Laor wrote:
> I agree that pci interface isn't ideal, the advantage is that platforms
> that do support pci don't have to implement a virtualized bus, and pci
> is PnP device. This was the motivation behind using it, especially for
> Windows.
In understand that requirement, and I think being able to surface 
paravirtual functions/devices to the guest as PCI device needs to be a 
key feature. For windows guests.

> Now that more platforms are joining the KVM wagon, we should define a
> common bus. PCI was a overkill anyway - its irq are shared and we don't
> have to use its io/mmio areas.
> Do you guys have something to start with?
We do have something basic, our vdev bus. The bad thing is, it is 
platform specific. I really prefer to aim for an idea that Arnd came 
up with when discussing this issue:
We could define a virtual device bus. We would have platform specific 
bus implementations which rely on hypercalls, interrupts, and shared 
memory in the pure paravirtual case. And we need a second bus driver, 
which is based on a pci device. Just similar to the situation where a 
PCI device is a USB host adapter.
Now functionality like device drivers can sit on top of the 
abstraction layer. We need to invent a similar abstraction on the 
hypervisor side, which is a tricky task indeed.
The best approach into this direction I have seen so far is not our 
own vdev thing but Rusty's virtio infrastructure. That's what I think 
I would start with.

so long,
Carsten

-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/

Re: Balloon driver?

[Avi Kivity]

Carsten Otte wrote:
> The best approach into this direction I have seen so far is not our 
> own vdev thing but Rusty's virtio infrastructure. That's what I think 
> I would start with.
>
>   

I agree.  We need a Lego set containing the following bricks:

- bus interfaces: pci bus, hypercall-based bus, xenbus
- functionality: net driver, block driver
- virtio data path: xen rings + grant tables + event channels, kvm rings
+ kmap_atomic + interrupts

So you can write a 4-line driver that glues the pci bus interface to the
net functionality driver to kvm virtio backend, and take the rest of the
day off.

-- 
Do not meddle in the internals of kernels, for they are subtle and quick to panic.


-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/

Re: Balloon driver?

[Arnd Bergmann]

On Wednesday 06 June 2007, Carsten Otte wrote:
> Dor Laor wrote:
>
> > Now that more platforms are joining the KVM wagon, we should define a
> > common bus. PCI was a overkill anyway - its irq are shared and we don't
> > have to use its io/mmio areas.
> > Do you guys have something to start with?
> We do have something basic, our vdev bus. The bad thing is, it is 
> platform specific. I really prefer to aim for an idea that Arnd came 
> up with when discussing this issue:
> We could define a virtual device bus.

I guess, more specifically, you mean a bus_type in Linux terminology.

> Now functionality like device drivers can sit on top of the 
> abstraction layer. We need to invent a similar abstraction on the 
> hypervisor side, which is a tricky task indeed.
> The best approach into this direction I have seen so far is not our 
> own vdev thing but Rusty's virtio infrastructure. That's what I think 
> I would start with.

So you suggest having a PCI driver that exports a virtio device for
each PCI device it matches? Sounds fine to me, though it goes beyond
what I think Rusty had in mind with his latest set of virtio
patches.

Note that contrary to what I may have said previously, I think that
such a virtio device on top of a PCI device need not use any PCI
specific APIs (readl/writel, pci_map_*, ioremap_, pci_dev->irq, ...)
but could do all of that by means of a hypercall interface. The
only real point we want to use PCI for is to find out about the
existence of a device and to get an identifier that gets passed
to the hypercalls.

	Arnd <><

-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/

Re: Balloon driver?

[Jun Koi]

On 6/7/07, Avi Kivity <avi-atKUWr5tajBWk0Htik3J/w@public.gmane.org> wrote:
> Carsten Otte wrote:
> > The best approach into this direction I have seen so far is not our
> > own vdev thing but Rusty's virtio infrastructure. That's what I think
> > I would start with.
> >
> >
>
> I agree.  We need a Lego set containing the following bricks:
>
> - bus interfaces: pci bus, hypercall-based bus, xenbus
> - functionality: net driver, block driver
> - virtio data path: xen rings + grant tables + event channels, kvm rings
> + kmap_atomic + interrupts
>

Avi, may you explain what is the role of the "bus interface" in the
whole picture? To autodetect and probe new virtual device??

About the balloon driver, I looked at the current code, and what I
dont like is its approach: it needs to be operated from inside the
guest, which is not the way we usually want to do. So something like
Xen balloon driver is certainly better.

Dor, why dont you get the idea from Xen balloon, but take your way?

Thanks,
Jun

-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/

Re: Balloon driver?

[Carsten Otte]

Arnd Bergmann wrote:
> On Wednesday 06 June 2007, Carsten Otte wrote:
>> We do have something basic, our vdev bus. The bad thing is, it is 
>> platform specific. I really prefer to aim for an idea that Arnd came 
>> up with when discussing this issue:
>> We could define a virtual device bus.
> 
> I guess, more specifically, you mean a bus_type in Linux terminology.
Indeed, sorry for the confusion.

> So you suggest having a PCI driver that exports a virtio device for
> each PCI device it matches? Sounds fine to me, though it goes beyond
> what I think Rusty had in mind with his latest set of virtio
> patches.
We don't need a PCI device per virtio device. We can have a PCI device 
that is a virtio bus driver.

> Note that contrary to what I may have said previously, I think that
> such a virtio device on top of a PCI device need not use any PCI
> specific APIs (readl/writel, pci_map_*, ioremap_, pci_dev->irq, ...)
> but could do all of that by means of a hypercall interface. The
> only real point we want to use PCI for is to find out about the
> existence of a device and to get an identifier that gets passed
> to the hypercalls.
True. We need to abstract how things get passed. Simple primitives, 
but good enough to provide all functionality needed by the exploiters. 
See Avis Lego box.

so long,
Carsten

-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/

Re: Balloon driver?

[Avi Kivity]

Jun Koi wrote:
> On 6/7/07, Avi Kivity <avi-atKUWr5tajBWk0Htik3J/w@public.gmane.org> wrote:
>> Carsten Otte wrote:
>> > The best approach into this direction I have seen so far is not our
>> > own vdev thing but Rusty's virtio infrastructure. That's what I think
>> > I would start with.
>> >
>> >
>>
>> I agree.  We need a Lego set containing the following bricks:
>>
>> - bus interfaces: pci bus, hypercall-based bus, xenbus
>> - functionality: net driver, block driver
>> - virtio data path: xen rings + grant tables + event channels, kvm rings
>> + kmap_atomic + interrupts
>>
>
> Avi, may you explain what is the role of the "bus interface" in the
> whole picture? To autodetect and probe new virtual device??
>

Yes.  I guess the most important part is to advertise the interrupt line
correctly to the guest OS, so it can apply its policies wrt sharing and
priorities as it wishes.  Hotplug/unplug and autodetection are also
important, but can probably be hacked even without pci.

-- 
Do not meddle in the internals of kernels, for they are subtle and quick to panic.


-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/

Re: Balloon driver?

[Avi Kivity]

Carsten Otte wrote:
>
>> So you suggest having a PCI driver that exports a virtio device for
>> each PCI device it matches? Sounds fine to me, though it goes beyond
>> what I think Rusty had in mind with his latest set of virtio
>> patches.
>>     
> We don't need a PCI device per virtio device. We can have a PCI device 
> that is a virtio bus driver.
>
>   

I think it is better to have a pci device per, er, device.  It's nice to
be able to do lspci (on an older Linux distro) and see what devices you
have, instead of requiring a new tool.  It will also not require us to
backport the virtual bus to older kernels.

Newer kernels can of course use a hypercall-based bus (suggestion: one
hypercall that reads an array of structures, each describing one device,
and one interrupt that means "please issue that hypercall and take
notice of any changes that may have occurred since the last time you did
that").



-- 
Do not meddle in the internals of kernels, for they are subtle and quick to panic.


-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/

Re: Balloon driver?

[Arnd Bergmann]

On Wednesday 06 June 2007, Jun Koi wrote:
> About the balloon driver, I looked at the current code, and what I
> dont like is its approach: it needs to be operated from inside the
> guest, which is not the way we usually want to do. So something like
> Xen balloon driver is certainly better.

There are multiple fundamentally different approaches. Three of
these have been used on s390 with success, and they are not
mutually exclusive:

1. Balloning: The guest voluntarily gives up pages using an
hcall if it has free pages or is told to do so by the
hypervisor.

2. pfault: The host takes away pages from the guest based on
LRU. When the guest accesses a page that has been paged out
by the host, it gets an exception and has the chance to switch
the guest task. Another exception is delivered when the page
is there, and the task gets marked runnable again.

3. "cmm2": The guest annotages every page as 'unused', 'stable',
'volatile' or 'potentially volatile', and the host memory
management code can take these into consideration. See
https://ols2006.108.redhat.com/reprints/schwidefsky-reprint.pdf
for details.

	Arnd <><

-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/

Re: Balloon driver?

[Rusty Russell]

On Wed, 2007-06-06 at 19:17 +0300, Avi Kivity wrote:
> Carsten Otte wrote:
> > The best approach into this direction I have seen so far is not our 
> > own vdev thing but Rusty's virtio infrastructure. That's what I think 
> > I would start with.
> >
> >   
> 
> I agree.  We need a Lego set containing the following bricks:
> 
> - bus interfaces: pci bus, hypercall-based bus, xenbus
> - functionality: net driver, block driver
> - virtio data path: xen rings + grant tables + event channels, kvm rings
> + kmap_atomic + interrupts
> 
> So you can write a 4-line driver that glues the pci bus interface to the
> net functionality driver to kvm virtio backend, and take the rest of the
> day off.

Speaking of which, I have been beavering away with various failed
attempts, and finally ate humble pie and am implementing the simple
callback scheme first suggested by Avi.  Everything else made the
drivers nasty and an efficient implementation nastier.

Expect draft II patches within 24 hours...
Rusty.




-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/

Re: Balloon driver?

[Dor Laor]

>> > Now that more platforms are joining the KVM wagon, we should define
a
>> > common bus. PCI was a overkill anyway - its irq are shared and we
don't
>> > have to use its io/mmio areas.
>> > Do you guys have something to start with?
>> We do have something basic, our vdev bus. The bad thing is, it is
>> platform specific. I really prefer to aim for an idea that Arnd came
>> up with when discussing this issue:
>> We could define a virtual device bus.
>
>I guess, more specifically, you mean a bus_type in Linux terminology.
>
>> Now functionality like device drivers can sit on top of the
>> abstraction layer. We need to invent a similar abstraction on the
>> hypervisor side, which is a tricky task indeed.
>> The best approach into this direction I have seen so far is not our
>> own vdev thing but Rusty's virtio infrastructure. That's what I think
>> I would start with.
>
>So you suggest having a PCI driver that exports a virtio device for
>each PCI device it matches? Sounds fine to me, though it goes beyond
>what I think Rusty had in mind with his latest set of virtio
>patches.
>
>Note that contrary to what I may have said previously, I think that
>such a virtio device on top of a PCI device need not use any PCI
>specific APIs (readl/writel, pci_map_*, ioremap_, pci_dev->irq, ...)
>but could do all of that by means of a hypercall interface. The
>only real point we want to use PCI for is to find out about the
>existence of a device and to get an identifier that gets passed
>to the hypercalls.

This is exactly our current approach. This is the way the current pv
network device is implemented and also the windows balloon driver.
Currently there is a small advantage of using io port/mmio because there
is no need to define additional shared page data, but that's negligible.


-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/