paravirtual drivers?

paravirtual drivers?

[Tomasz Chmielewski]

As I understand, fully virtualized KVM guests (on Intel VT or AMD-V) can 
perform with near native speeds only when it comes to the CPU - because 
the things like network or block device are basically emulated.


Among many changes, KVM in 2.6.21 has "Initial paravirtualization 
support, which has much faster performance", and is based on KVM-15.


At least two PV drivers are needed:
- network
- block device


What is the state of these PV drivers now? Is it usable on Linux now?

Are Windows PV drivers planned, too? If yes, who is going to take the 
lead here (Qumranet, community, not decided yet etc.?).



-- 
Tomasz Chmielewski
http://wpkg.org



-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

Re: paravirtual drivers?

[Avi Kivity]

Tomasz Chmielewski wrote:
> As I understand, fully virtualized KVM guests (on Intel VT or AMD-V) can 
> perform with near native speeds only when it comes to the CPU - because 
> the things like network or block device are basically emulated.
>
>
> Among many changes, KVM in 2.6.21 has "Initial paravirtualization 
> support, which has much faster performance", and is based on KVM-15.
>
>
> At least two PV drivers are needed:
> - network
> - block device
>
>
> What is the state of these PV drivers now? Is it usable on Linux now?
>   

We (Qumranet) have Linux paravirtual network support working, but not 
ready for general consumption yet.

> Are Windows PV drivers planned, too? If yes, who is going to take the 
> lead here (Qumranet, community, not decided yet etc.?).
>   

We also have Windows network drivers.

-- 
error compiling committee.c: too many arguments to function


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

Re: paravirtual drivers?

[Dor Laor]

>As I understand, fully virtualized KVM guests (on Intel VT or AMD-V)
can
>perform with near native speeds only when it comes to the CPU - because
>the things like network or block device are basically emulated.
>
>
>Among many changes, KVM in 2.6.21 has "Initial paravirtualization
>support, which has much faster performance", and is based on KVM-15.
>
>
>At least two PV drivers are needed:
>- network
>- block device
>
>
>What is the state of these PV drivers now? Is it usable on Linux now?

The network PV driver that was initiated by Ingo Molnar is developed by
Yan Vugenfirer and me.
It is not yet stable/clean for main stream although can be checked out
from a private git repo.
Using direct irq injection it can reach multi-gigabit throughput.
Using signals sent to qemu (to sync with APIC's tpr) it does only
~500Mbps.
When the APIC/PIC will reside in the kernel we can pass to direct mode
only.
Expect to publish the driver in the May.

Arjan van de Ven from RedHat is working on pv block device.
Status is unknown.

>
>Are Windows PV drivers planned, too? If yes, who is going to take the
>lead here (Qumranet, community, not decided yet etc.?).
>
>
>
>--
>Tomasz Chmielewski
>http://wpkg.org
>
>
>
>-----------------------------------------------------------------------
--
>Take Surveys. Earn Cash. Influence the Future of IT
>Join SourceForge.net's Techsay panel and you'll get the chance to share
>your
>opinions on IT & business topics through brief surveys-and earn cash
>http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVD
EV
>_______________________________________________
>kvm-devel mailing list
>kvm-devel-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org
>https://lists.sourceforge.net/lists/listinfo/kvm-devel

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

Re: paravirtual drivers?

[Tomasz Chmielewski]

Avi Kivity schrieb:
> Tomasz Chmielewski wrote:
>> As I understand, fully virtualized KVM guests (on Intel VT or AMD-V) 
>> can perform with near native speeds only when it comes to the CPU - 
>> because the things like network or block device are basically emulated.
>>
>>
>> Among many changes, KVM in 2.6.21 has "Initial paravirtualization 
>> support, which has much faster performance", and is based on KVM-15.
>>
>>
>> At least two PV drivers are needed:
>> - network
>> - block device
>>
>>
>> What is the state of these PV drivers now? Is it usable on Linux now?
>>   
> 
> We (Qumranet) have Linux paravirtual network support working, but not 
> ready for general consumption yet.

Hmm, is the "general consumption" of the network support a matter of 
weeks, or months rather?
And what about the block device? As I understand, using SCSI emulation 
gives better results, but still, there will be a noticeable difference 
when compared to the "native speed".


>> Are Windows PV drivers planned, too? If yes, who is going to take the 
>> lead here (Qumranet, community, not decided yet etc.?).
>>   
> 
> We also have Windows network drivers.

Is it decided if these PV drivers will be open source/free, or rather 
have commercial model?


-- 
Tomasz Chmielewski
http://wpkg.org

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

Re: paravirtual drivers?

[Rami Tamir]

Tomasz Chmielewski wrote:
> Is it decided if these PV drivers will be open source/free, or rather 
> have commercial model?

We are planning to open source all the drivers but we are still looking at the legal aspects of opening a MS DDK based code, hopefully we'll have answers shortly. The Windows binaries will be free and available once they are ready for public consumption.

Rami
-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

Re: paravirtual drivers?

[Avi Kivity]

Tomasz Chmielewski wrote:
>>>
>>> What is the state of these PV drivers now? Is it usable on Linux now?
>>>   
>>
>> We (Qumranet) have Linux paravirtual network support working, but not 
>> ready for general consumption yet.
>
> Hmm, is the "general consumption" of the network support a matter of 
> weeks, or months rather?

Hopefully it will be included in kvm releases in a few weeks, but 
perhaps not as efficient as we'd like, and it would be without 
commitment to ABI stability (i.e. you may have to install a new guest 
driver after updating the host).

An ABI-stable driver will be available on a longer time frame.  We're 
still hashing out things like where the host-side driver belongs.  There 
are known performance issues with the current driver (though it's 
magnitudes better than the qemu devices).  I'm toying with the idea of 
sharing the Xen drivers.

> And what about the block device? As I understand, using SCSI emulation 
> gives better results, but still, there will be a noticeable difference 
> when compared to the "native speed".
>

We're not working on block devices at the moment, and I have no 
performance figures on scsi vs. ide vs. paravirt.


-- 
error compiling committee.c: too many arguments to function


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

Re: paravirtual drivers?

[Arjan van de Ven]

> When the APIC/PIC will reside in the kernel we can pass to direct mode
> only.
> Expect to publish the driver in the May.
> 
> Arjan van de Ven from RedHat is working on pv block device.
> Status is unknown.
> 

I'm sorry but I'm from Intel, not Red Hat.

the pv block driver is in Ingo's -rt tree if you want to look at it.


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

Re: paravirtual drivers?

[Dor Laor]

>> When the APIC/PIC will reside in the kernel we can pass to direct
mode
>> only.
>> Expect to publish the driver in the May.
>>
>> Arjan van de Ven from RedHat is working on pv block device.
>> Status is unknown.
>>
>
>I'm sorry but I'm from Intel, not Red Hat.

Opss, I thought you and Ingo work for the same company.
I guess email cover the distance/vendor gaps.

>
>the pv block driver is in Ingo's -rt tree if you want to look at it.

Glad to know you made it to his tree :)
When do you think it can be integrated to mainline kvm?

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

Re: paravirtual drivers?

[Anthony Liguori]

Tomasz Chmielewski wrote:
> As I understand, fully virtualized KVM guests (on Intel VT or AMD-V) can 
> perform with near native speeds only when it comes to the CPU - because 
> the things like network or block device are basically emulated.
>
>
> Among many changes, KVM in 2.6.21 has "Initial paravirtualization 
> support, which has much faster performance", and is based on KVM-15.
>
>
> At least two PV drivers are needed:
> - network
> - block device
>   

I don't believe a PV block driver is actually necessary.  Disk devices 
are actually quite slow (unless you have a large array).  With a few 
small changes (bump MMU cache up to 1024 and actually enable the aio 
subsystem), SCSI emulation does as well as Xen's PV block driver (about 
210 mb/sec whereas native is around 250 mb/sec).  There's a lot of room 
to optimize the SCSI emulation too since there are a number of copies in 
that path.

The SCSI emulation is a lot like what one would design in a PV driver.  
It supports multiple outstanding requests and doesn't require a whole 
lot of exits to submit a request (it uses scripts).

Regards,

Anthony Liguori

>
> What is the state of these PV drivers now? Is it usable on Linux now?
>
> Are Windows PV drivers planned, too? If yes, who is going to take the 
> lead here (Qumranet, community, not decided yet etc.?).
>
>
>
>   


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

Re: paravirtual drivers?

[Arjan van de Ven]

> >
> >the pv block driver is in Ingo's -rt tree if you want to look at it.
> 
> Glad to know you made it to his tree :)
> When do you think it can be integrated to mainline kvm?

the biggest gap left is how to associate the virtual file with the host.
Once I've figured that it's ready to go imo.
(doesn't mean it can't be improved further, esp on the performance side
of the backend things can be improved, but the interfaces and client
side should be basically finished)
-- 
if you want to mail me at work (you don't), use arjan (at) linux.intel.com
Test the interaction between Linux and your BIOS via http://www.linuxfirmwarekit.org


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

Re: paravirtual drivers?

[Anthony Liguori]

Arjan van de Ven wrote:
>>> the pv block driver is in Ingo's -rt tree if you want to look at it.
>>>       
>> Glad to know you made it to his tree :)
>> When do you think it can be integrated to mainline kvm?
>>     
>
> the biggest gap left is how to associate the virtual file with the host.
>   

Why not drive the backend from userspace?  Are you using a ring queue to 
transmit bio's from the guest to the host?  What are you using for 
discovery (I assume PCI)?

Could you post a copy of your driver on the list?  I think it would be 
good to discuss.

Regards,

Anthony Liguori

> Once I've figured that it's ready to go imo.
> (doesn't mean it can't be improved further, esp on the performance side
> of the backend things can be improved, but the interfaces and client
> side should be basically finished)
>   


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

Re: paravirtual drivers?

[Avi Kivity]

Arjan van de Ven wrote:
> the biggest gap left is how to associate the virtual file with the host.
> Once I've figured that it's ready to go imo.
>   

You mean connecting the backend with the host file?  Maybe an ioctl() 
with an fd argument?


-- 
error compiling committee.c: too many arguments to function


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

Re: paravirtual drivers?

[Arjan van de Ven]

On Thu, 2007-04-12 at 10:05 -0500, Anthony Liguori wrote:
> Arjan van de Ven wrote:
> >>> the pv block driver is in Ingo's -rt tree if you want to look at it.
> >>>       
> >> Glad to know you made it to his tree :)
> >> When do you think it can be integrated to mainline kvm?
> >>     
> >
> > the biggest gap left is how to associate the virtual file with the host.
> >   
> 
> Why not drive the backend from userspace? 

(nothing in the concept requires it to be in the kernel, it just makes
things a lot easier)

Because I think the trend forward is to reduce the userspace dependency
of kvm, not increase it. And because you add context switches etc etc

>  Are you using a ring queue to 
> transmit bio's from the guest to the host? 

sort of; it's not quite a real ring (I'm using the block layer tagged
queueing which is more efficient than a ring actually)

>  What are you using for 
> discovery (I assume PCI)?

I just added a hypercall to enumerate. Adding a dummy PCI is probably
needed for OS installers to auto-load the drivers.

> 
> Could you post a copy of your driver on the list?  I think it would be 
> good to discuss.

I'll dig it up
-- 
if you want to mail me at work (you don't), use arjan (at) linux.intel.com
Test the interaction between Linux and your BIOS via http://www.linuxfirmwarekit.org


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

Re: paravirtual drivers?

[Anthony Liguori]

Arjan van de Ven wrote:
> On Thu, 2007-04-12 at 10:05 -0500, Anthony Liguori wrote:
>   
>> Why not drive the backend from userspace? 
>>     
>
> (nothing in the concept requires it to be in the kernel, it just makes
> things a lot easier)
>
> Because I think the trend forward is to reduce the userspace dependency
> of kvm, not increase it.

There's no increase in dependency since block IO is already done in 
userspace.  Besides, I think the general Linux trend of moving things to 
userspace when there's no compelling reason to have them in kernel space 
trumps all other trends ;-)

>  And because you add context switches etc etc
>   

There's no context switch just a privilege transition.  Granted, 
privilege transitions aren't free but they're aren't that much compared 
to the cost of processing an IO request.

>>  Are you using a ring queue to 
>> transmit bio's from the guest to the host? 
>>     
>
> sort of; it's not quite a real ring (I'm using the block layer tagged
> queueing which is more efficient than a ring actually)
>   

I'm not sure I understand.  I guess I can just read the code.  What I 
was wondering was whether you put the bio's into a shared ring queue or 
whether you issued a hypercall per bio.

>>  What are you using for 
>> discovery (I assume PCI)?
>>     
>
> I just added a hypercall to enumerate. Adding a dummy PCI is probably
> needed for OS installers to auto-load the drivers.
>   

Ah, okay.

>> Could you post a copy of your driver on the list?  I think it would be 
>> good to discuss.
>>     
>
> I'll dig it up
>   

Excellent.  Thank you!

Regards,

Anthony Liguori


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

Re: paravirtual drivers?

[Tomasz Chmielewski]

Anthony Liguori schrieb:

(...)

> I don't believe a PV block driver is actually necessary.  Disk devices 
> are actually quite slow (unless you have a large array).  With a few 
> small changes (bump MMU cache up to 1024 and actually enable the aio 
> subsystem), SCSI emulation does as well as Xen's PV block driver (about 
> 210 mb/sec whereas native is around 250 mb/sec).  There's a lot of room 
> to optimize the SCSI emulation too since there are a number of copies in 
> that path.
> 
> The SCSI emulation is a lot like what one would design in a PV driver.  
> It supports multiple outstanding requests and doesn't require a whole 
> lot of exits to submit a request (it uses scripts).

How does one enable SCSI emulation?

"qemu-system-x86_64 -h" gives no hints about SCSI (at least in KVM-18); 
similarly, I didn't find anything in the archive or in the wiki.


Certainly, there is a kvm-18/qemu/hw/scsi-disk.c file, so it's like the 
feature is there...

Is it possible to boot a guest from a "SCSI-emulated" device?


-- 
Tomasz Chmielewski
http://wpkg.org

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

Re: paravirtual drivers?

[Anthony Liguori]

Tomasz Chmielewski wrote:
> Anthony Liguori schrieb:
>
> (...)
>
>> I don't believe a PV block driver is actually necessary.  Disk 
>> devices are actually quite slow (unless you have a large array).  
>> With a few small changes (bump MMU cache up to 1024 and actually 
>> enable the aio subsystem), SCSI emulation does as well as Xen's PV 
>> block driver (about 210 mb/sec whereas native is around 250 mb/sec).  
>> There's a lot of room to optimize the SCSI emulation too since there 
>> are a number of copies in that path.
>>
>> The SCSI emulation is a lot like what one would design in a PV 
>> driver.  It supports multiple outstanding requests and doesn't 
>> require a whole lot of exits to submit a request (it uses scripts).
>
> How does one enable SCSI emulation?

Right now, you have to modify the source code :-/

> "qemu-system-x86_64 -h" gives no hints about SCSI (at least in 
> KVM-18); similarly, I didn't find anything in the archive or in the wiki.
>
>
> Certainly, there is a kvm-18/qemu/hw/scsi-disk.c file, so it's like 
> the feature is there...
>
> Is it possible to boot a guest from a "SCSI-emulated" device?

Not yet.  There is work being done on an option ROM to allow SCSI boot 
support though.

Regards,

Anthony Liguori


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV