[PATCH] Make virtio devices multi-function

[PATCH] Make virtio devices multi-function

[Anthony Liguori]

This patch changes virtio devices to be multi-function devices whenever
possible.  This increases the number of virtio devices we can support now by
a factor of 8.

With this patch, I've been able to launch a guest with either 220 disks or 220
network adapters.

I haven't tested the Windows virtio drivers.

Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>

diff --git a/qemu/hw/pci.h b/qemu/hw/pci.h
index 60e4094..df3a878 100644
--- a/qemu/hw/pci.h
+++ b/qemu/hw/pci.h
@@ -33,7 +33,7 @@ typedef struct PCIIORegion {
 #define PCI_ROM_SLOT 6
 #define PCI_NUM_REGIONS 7
 
-#define PCI_DEVICES_MAX 64
+#define PCI_DEVICES_MAX 256
 
 #define PCI_VENDOR_ID		0x00	/* 16 bits */
 #define PCI_DEVICE_ID		0x02	/* 16 bits */
diff --git a/qemu/hw/virtio.c b/qemu/hw/virtio.c
index 9100bb1..9ea14d3 100644
--- a/qemu/hw/virtio.c
+++ b/qemu/hw/virtio.c
@@ -405,9 +405,18 @@ VirtIODevice *virtio_init_pci(PCIBus *bus, const char *name,
     PCIDevice *pci_dev;
     uint8_t *config;
     uint32_t size;
+    static int devfn = 7;
+
+    if ((devfn % 8) == 7)
+	devfn = -1;
+    else
+	devfn++;
 
     pci_dev = pci_register_device(bus, name, struct_size,
-				  -1, NULL, NULL);
+				  devfn, NULL, NULL);
+
+    devfn = pci_dev->devfn;
+
     vdev = to_virtio_device(pci_dev);
 
     vdev->status = 0;
@@ -435,6 +444,10 @@ VirtIODevice *virtio_init_pci(PCIBus *bus, const char *name,
 
     config[0x3d] = 1;
 
+    /* Mark device as multi-function */
+    if ((devfn % 8) == 0)
+	config[0x0e] |= 0x80;
+
     vdev->name = name;
     vdev->config_len = config_size;
     if (vdev->config_len)
diff --git a/qemu/net.h b/qemu/net.h
index 13daa27..3bada75 100644
--- a/qemu/net.h
+++ b/qemu/net.h
@@ -42,7 +42,7 @@ void net_client_uninit(NICInfo *nd);
 
 /* NIC info */
 
-#define MAX_NICS 8
+#define MAX_NICS 256
 
 struct NICInfo {
     uint8_t macaddr[6];
diff --git a/qemu/sysemu.h b/qemu/sysemu.h
index b645fb7..7992a77 100644
--- a/qemu/sysemu.h
+++ b/qemu/sysemu.h
@@ -151,7 +151,7 @@ typedef struct DriveInfo {
 
 #define MAX_IDE_DEVS	2
 #define MAX_SCSI_DEVS	7
-#define MAX_DRIVES 32
+#define MAX_DRIVES 256
 
 int nb_drives;
 DriveInfo drives_table[MAX_DRIVES+1];
diff --git a/qemu/vl.c b/qemu/vl.c
index 7dd0094..e203a4d 100644
--- a/qemu/vl.c
+++ b/qemu/vl.c
@@ -8754,7 +8754,7 @@ static BOOL WINAPI qemu_ctrl_handler(DWORD type)
 }
 #endif
 
-#define MAX_NET_CLIENTS 32
+#define MAX_NET_CLIENTS 512
 
 static int saved_argc;
 static char **saved_argv;

-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference 
Don't miss this year's exciting event. There's still time to save $100. 
Use priority code J8TL2D2. 
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone

Re: [PATCH] Make virtio devices multi-function

[Avi Kivity]

Anthony Liguori wrote:
> This patch changes virtio devices to be multi-function devices whenever
> possible.  This increases the number of virtio devices we can support now by
> a factor of 8.
>
> With this patch, I've been able to launch a guest with either 220 disks or 220
> network adapters.
>
>   

Does this play well with hotplug?  Perhaps we need to allocate a new 
device on hotplug.

(certainly if we have a device with one function, which then gets 
converted to a multifunction device)

-- 
error compiling committee.c: too many arguments to function


-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference 
Don't miss this year's exciting event. There's still time to save $100. 
Use priority code J8TL2D2. 
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone

Re: [PATCH] Make virtio devices multi-function

[Anthony Liguori]

Avi Kivity wrote:
> Anthony Liguori wrote:
>   
>> This patch changes virtio devices to be multi-function devices whenever
>> possible.  This increases the number of virtio devices we can support now by
>> a factor of 8.
>>
>> With this patch, I've been able to launch a guest with either 220 disks or 220
>> network adapters.
>>
>>   
>>     
>
> Does this play well with hotplug?  Perhaps we need to allocate a new 
> device on hotplug.
>   

Probably not.  I imagine you can only hotplug devices, not individual 
functions?

Regards,

Anthony Liguori

> (certainly if we have a device with one function, which then gets 
> converted to a multifunction device)
>
>   


-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference 
Don't miss this year's exciting event. There's still time to save $100. 
Use priority code J8TL2D2. 
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone

Re: [PATCH] Make virtio devices multi-function

[Avi Kivity]

Anthony Liguori wrote:
> Avi Kivity wrote:
>> Anthony Liguori wrote:
>>  
>>> This patch changes virtio devices to be multi-function devices whenever
>>> possible.  This increases the number of virtio devices we can 
>>> support now by
>>> a factor of 8.
>>>
>>> With this patch, I've been able to launch a guest with either 220 
>>> disks or 220
>>> network adapters.
>>>
>>>       
>>
>> Does this play well with hotplug?  Perhaps we need to allocate a new 
>> device on hotplug.
>>   
>
> Probably not.  I imagine you can only hotplug devices, not individual 
> functions?
>

It sounds reasonable to expect so.  ACPI has objects for devices, not 
functions (IIRC).

Maybe require explicit device/function assignment on the command line?  
It will be managed anyway.

-- 
error compiling committee.c: too many arguments to function


-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference 
Don't miss this year's exciting event. There's still time to save $100. 
Use priority code J8TL2D2. 
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone

Re: [PATCH] Make virtio devices multi-function

[Marcelo Tosatti]

On Tue, Apr 22, 2008 at 05:51:51PM +0300, Avi Kivity wrote:
> Anthony Liguori wrote:
> > Avi Kivity wrote:
> >> Anthony Liguori wrote:
> >>  
> >>> This patch changes virtio devices to be multi-function devices whenever
> >>> possible.  This increases the number of virtio devices we can 
> >>> support now by
> >>> a factor of 8.
> >>>
> >>> With this patch, I've been able to launch a guest with either 220 
> >>> disks or 220
> >>> network adapters.
> >>>
> >>>       
> >>
> >> Does this play well with hotplug?  Perhaps we need to allocate a new 
> >> device on hotplug.
> >>   
> >
> > Probably not.  I imagine you can only hotplug devices, not individual 
> > functions?
> >
> 
> It sounds reasonable to expect so.  ACPI has objects for devices, not 
> functions (IIRC).

So what I dislike about multifunction devices is the fact that a single
slot shares an IRQ, and that special code is required in the QEMU
drivers (virtio guest capability might not always be present).

I don't see any need for using them if we can extend PCI slots...

> Maybe require explicit device/function assignment on the command line?  
> It will be managed anyway.

ACPI does support hotplugging of individual functions inside slots,
not sure how well does Linux (and other OSes) support that.. should be
transparent though.



-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference 
Don't miss this year's exciting event. There's still time to save $100. 
Use priority code J8TL2D2. 
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone

Re: [PATCH] Make virtio devices multi-function

[Anthony Liguori]

Marcelo Tosatti wrote:
>> Maybe require explicit device/function assignment on the command line?  
>> It will be managed anyway.
>>     
>
> ACPI does support hotplugging of individual functions inside slots,
> not sure how well does Linux (and other OSes) support that.. should be
> transparent though.
>   

I think we need to decide what we want to target in terms of upper limits.

With a bridge or two, we can probably easily do 128.

If we really want to push things, I think we should do a PCI based 
virtio controller.  I doubt a large number of PCI devices is ever going 
to perform very well b/c of interrupt sharing and some of the 
assumptions in virtio_pci.

If we implement a controller, we can use a single interrupt, but 
multiplex multiple notifications on that single interrupt.  We can also 
be more aggressive about using shared memory instead of PCI config space 
which would reduce the overall number of exits.

We could easily support a very large number of devices this way.  But 
again, what do we want to target for now?

Regards,

Anthony Liguori



-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference 
Don't miss this year's exciting event. There's still time to save $100. 
Use priority code J8TL2D2. 
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone

Re: [PATCH] Make virtio devices multi-function

[Avi Kivity]

Anthony Liguori wrote:
>
> I think we need to decide what we want to target in terms of upper 
> limits.
>
> With a bridge or two, we can probably easily do 128.
>
> If we really want to push things, I think we should do a PCI based 
> virtio controller.  I doubt a large number of PCI devices is ever 
> going to perform very well b/c of interrupt sharing and some of the 
> assumptions in virtio_pci.
>
> If we implement a controller, we can use a single interrupt, but 
> multiplex multiple notifications on that single interrupt.  We can 
> also be more aggressive about using shared memory instead of PCI 
> config space which would reduce the overall number of exits.
>
> We could easily support a very large number of devices this way.  But 
> again, what do we want to target for now? 

I think that for networking we should keep things as is.  I don't see 
anybody using 100 virtual NICs.

For mass storage, we should follow the SCSI model with a single device 
serving multiple disks, similar to what you suggest.  Not sure if the 
device should have a single queue or one queue per disk.

-- 
error compiling committee.c: too many arguments to function


-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference 
Don't miss this year's exciting event. There's still time to save $100. 
Use priority code J8TL2D2. 
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone

Re: [PATCH] Make virtio devices multi-function

[Anthony Liguori]

Avi Kivity wrote:
> Anthony Liguori wrote:
>>
>> I think we need to decide what we want to target in terms of upper 
>> limits.
>>
>> With a bridge or two, we can probably easily do 128.
>>
>> If we really want to push things, I think we should do a PCI based 
>> virtio controller.  I doubt a large number of PCI devices is ever 
>> going to perform very well b/c of interrupt sharing and some of the 
>> assumptions in virtio_pci.
>>
>> If we implement a controller, we can use a single interrupt, but 
>> multiplex multiple notifications on that single interrupt.  We can 
>> also be more aggressive about using shared memory instead of PCI 
>> config space which would reduce the overall number of exits.
>>
>> We could easily support a very large number of devices this way.  But 
>> again, what do we want to target for now? 
>
> I think that for networking we should keep things as is.  I don't see 
> anybody using 100 virtual NICs.
>
> For mass storage, we should follow the SCSI model with a single device 
> serving multiple disks, similar to what you suggest.  Not sure if the 
> device should have a single queue or one queue per disk.

My latest thought it to do a virtio-based virtio controller.

We could avoid creating one in QEMU unless we detect an abnormally large 
number of disks or something.

Regards,

Anthony Liguori


-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference 
Don't miss this year's exciting event. There's still time to save $100. 
Use priority code J8TL2D2. 
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone

Re: [PATCH] Make virtio devices multi-function

[Marcelo Tosatti]

On Tue, Apr 22, 2008 at 11:31:11AM -0500, Anthony Liguori wrote:
> Avi Kivity wrote:
> >Anthony Liguori wrote:
> >>
> >>I think we need to decide what we want to target in terms of upper 
> >>limits.
> >>
> >>With a bridge or two, we can probably easily do 128.
> >>
> >>If we really want to push things, I think we should do a PCI based 
> >>virtio controller.  I doubt a large number of PCI devices is ever 
> >>going to perform very well b/c of interrupt sharing and some of the 
> >>assumptions in virtio_pci.
>>
> >>If we implement a controller, we can use a single interrupt, but 
> >>multiplex multiple notifications on that single interrupt.  We can 
> >>also be more aggressive about using shared memory instead of PCI 
> >>config space which would reduce the overall number of exits.

We should increase the number of interrupt lines, perhaps to 16.

Using shared memory to avoid exits sounds very good idea.

> >>We could easily support a very large number of devices this way.  But 
> >>again, what do we want to target for now? 
> >
> >I think that for networking we should keep things as is.  I don't see 
> >anybody using 100 virtual NICs.

The target was along the lines of 20 nics + 80 disks. Dan?

> >For mass storage, we should follow the SCSI model with a single device 
> >serving multiple disks, similar to what you suggest.  Not sure if the 
> >device should have a single queue or one queue per disk.
> 
> My latest thought it to do a virtio-based virtio controller.

Why do you dislike multiple disks per virtio-blk controller? As
mentioned this seems a natural way forward.

-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference 
Don't miss this year's exciting event. There's still time to save $100. 
Use priority code J8TL2D2. 
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone

Re: [PATCH] Make virtio devices multi-function

[Anthony Liguori]

>>> For mass storage, we should follow the SCSI model with a single device 
>>> serving multiple disks, similar to what you suggest.  Not sure if the 
>>> device should have a single queue or one queue per disk.
>>>       
>> My latest thought it to do a virtio-based virtio controller.
>>     
>
> Why do you dislike multiple disks per virtio-blk controller? As
> mentioned this seems a natural way forward.
>   

Logically speaking, virtio is a bus.  virtio supports all of the 
features of a bus (discover, hot add, hot remove).

Right now, we map virtio devices directly onto the PCI bus.

The problem we're trying to address is limitations of the PCI bus.  We 
have a couple options:

1) add a virtio device that supports multiple disks.  we need to 
reinvent hotplug within this device.

2) add a new PCI virtio transport that supports multiple virtio-blk 
devices within a single PCI slot

3) add a generic PCI virtio transport that supports multiple virtio 
devices within a single PCI slot

4) add a generic virtio "bridge" that supports multiple virtio devices 
within a single virtio device.

#4 may seem strange, but it's no different from a PCI-to-PCI bridge.

I like #4 the most, but #2 is probably the most practical.


Regards,

Anthony Liguori

-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference 
Don't miss this year's exciting event. There's still time to save $100. 
Use priority code J8TL2D2. 
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone

Re: [PATCH] Make virtio devices multi-function

[Chris Wright]

* Anthony Liguori (anthony@codemonkey.ws) wrote:
> Logically speaking, virtio is a bus.  virtio supports all of the features 
> of a bus (discover, hot add, hot remove).
>
> Right now, we map virtio devices directly onto the PCI bus.
>
> The problem we're trying to address is limitations of the PCI bus.  We have 
> a couple options:

First question is if we have a real limitation with multiple busses?

> 1) add a virtio device that supports multiple disks.  we need to reinvent 
> hotplug within this device.
>
> 2) add a new PCI virtio transport that supports multiple virtio-blk devices 
> within a single PCI slot
>
> 3) add a generic PCI virtio transport that supports multiple virtio devices 
> within a single PCI slot

compare and contrast above with HBA and disks (makes most sense from my
point of view).  for 2 and 3, only difference is whether you want to be
able to support nics, balloons, and block devices on same pci slot (at
which point it's a bridge, how is it different from 4?)

> 4) add a generic virtio "bridge" that supports multiple virtio devices 
> within a single virtio device.
>
> #4 may seem strange, but it's no different from a PCI-to-PCI bridge.
>
> I like #4 the most, but #2 is probably the most practical.

Also, your current patch does not work for hotplug disk.

thanks,
-chris

-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference 
Don't miss this year's exciting event. There's still time to save $100. 
Use priority code J8TL2D2. 
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone

Re: [PATCH] Make virtio devices multi-function

[Daniel P. Berrange]

On Tue, Apr 22, 2008 at 02:26:45PM -0300, Marcelo Tosatti wrote:
> On Tue, Apr 22, 2008 at 11:31:11AM -0500, Anthony Liguori wrote:
> > Avi Kivity wrote:
> > >Anthony Liguori wrote:
> > >>
> > >>I think we need to decide what we want to target in terms of upper 
> > >>limits.
> > >>
> > >>With a bridge or two, we can probably easily do 128.
> > >>
> > >>If we really want to push things, I think we should do a PCI based 
> > >>virtio controller.  I doubt a large number of PCI devices is ever 
> > >>going to perform very well b/c of interrupt sharing and some of the 
> > >>assumptions in virtio_pci.
> >>
> > >>If we implement a controller, we can use a single interrupt, but 
> > >>multiplex multiple notifications on that single interrupt.  We can 
> > >>also be more aggressive about using shared memory instead of PCI 
> > >>config space which would reduce the overall number of exits.
> 
> We should increase the number of interrupt lines, perhaps to 16.
> 
> Using shared memory to avoid exits sounds very good idea.
> 
> > >>We could easily support a very large number of devices this way.  But 
> > >>again, what do we want to target for now? 
> > >
> > >I think that for networking we should keep things as is.  I don't see 
> > >anybody using 100 virtual NICs.
> 
> The target was along the lines of 20 nics + 80 disks. Dan?

I've already had people ask for ability to as many as 64 disks and 32 nics
with Xen, so to my mind, the more we support the better. 100's if possible.

Dan.
-- 
|: Red Hat, Engineering, Boston   -o-   http://people.redhat.com/berrange/ :|
|: http://libvirt.org  -o-  http://virt-manager.org  -o-  http://ovirt.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505  -o-  F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|

-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference 
Don't miss this year's exciting event. There's still time to save $100. 
Use priority code J8TL2D2. 
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone

Re: [PATCH] Make virtio devices multi-function

[Ian Kirk]

Avi Kivity wrote:

> For mass storage, we should follow the SCSI model with a single device
> serving multiple disks, similar to what you suggest.  Not sure if the
> device should have a single queue or one queue per disk.

Don't you just end up re-implementing SCSI then, at which point you might
as well stick with a 'fake' SCSI device in the guest?

-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference 
Don't miss this year's exciting event. There's still time to save $100. 
Use priority code J8TL2D2. 
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone

Re: [PATCH] Make virtio devices multi-function

[Avi Kivity]

Ian Kirk wrote:
> Avi Kivity wrote:
>
>   
>> For mass storage, we should follow the SCSI model with a single device
>> serving multiple disks, similar to what you suggest.  Not sure if the
>> device should have a single queue or one queue per disk.
>>     
>
> Don't you just end up re-implementing SCSI then, at which point you might
> as well stick with a 'fake' SCSI device in the guest?
>   

A virtio-scsi controller is indeed useful as it can control tapes, media 
changers, and other fancy stuff in addition to ordinary disks.  For 
disks, I'd like to avoid the overhead of scsi command generation and 
parsing.

-- 
Any sufficiently difficult bug is indistinguishable from a feature.


-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference 
Don't miss this year's exciting event. There's still time to save $100. 
Use priority code J8TL2D2. 
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone

Re: [PATCH] Make virtio devices multi-function

[Marcelo Tosatti]

> > It sounds reasonable to expect so.  ACPI has objects for devices, not 
> > functions (IIRC).
> 
> So what I dislike about multifunction devices is the fact that a single
> slot shares an IRQ, and that special code is required in the QEMU
> drivers (virtio guest capability might not always be present).

Actually this is not true. It is possible to have separate IRQ's for
different devices residing in the same slot, changes are required to the
ACPI tables though.

> I don't see any need for using them if we can extend PCI slots...
> 
> > Maybe require explicit device/function assignment on the command line?  
> > It will be managed anyway.
> 
> ACPI does support hotplugging of individual functions inside slots,
> not sure how well does Linux (and other OSes) support that.. should be
> transparent though.


-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference 
Don't miss this year's exciting event. There's still time to save $100. 
Use priority code J8TL2D2. 
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone

Re: [PATCH] Make virtio devices multi-function

[Marcelo Tosatti]

On Tue, Apr 22, 2008 at 05:32:45PM +0300, Avi Kivity wrote:
> Anthony Liguori wrote:
> > This patch changes virtio devices to be multi-function devices whenever
> > possible.  This increases the number of virtio devices we can support now by
> > a factor of 8.
> >
> > With this patch, I've been able to launch a guest with either 220 disks or 220
> > network adapters.
> >
> >   
> 
> Does this play well with hotplug?  Perhaps we need to allocate a new 
> device on hotplug.
> 
> (certainly if we have a device with one function, which then gets 
> converted to a multifunction device)

Would have to change the hotplug code to handle functions...

It sounds less hacky to just extend the PCI slots instead of (ab)using
multiple functions per-slot.

-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference 
Don't miss this year's exciting event. There's still time to save $100. 
Use priority code J8TL2D2. 
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone

Re: [PATCH] Make virtio devices multi-function

[Anthony Liguori]

Marcelo Tosatti wrote:
> On Tue, Apr 22, 2008 at 05:32:45PM +0300, Avi Kivity wrote:
>   
>> Anthony Liguori wrote:
>>     
>>> This patch changes virtio devices to be multi-function devices whenever
>>> possible.  This increases the number of virtio devices we can support now by
>>> a factor of 8.
>>>
>>> With this patch, I've been able to launch a guest with either 220 disks or 220
>>> network adapters.
>>>
>>>   
>>>       
>> Does this play well with hotplug?  Perhaps we need to allocate a new 
>> device on hotplug.
>>
>> (certainly if we have a device with one function, which then gets 
>> converted to a multifunction device)
>>     
>
> Would have to change the hotplug code to handle functions...
>   

BTW, I've never been that convinced that hotplugging devices is as 
useful as people make it out to be.  I also think that's particularly 
true when it comes to hot adding/removing very large numbers of disks.

I think if you created all virtio devices as multifunction devices, but 
didn't add additional functions until you ran out of PCI slots, it would 
be a pretty acceptable solution.  Hotplug works just as it does today 
until you get much higher than 32 devices.  Even then, hotplug still 
works with most of your devices (until you hit the absolute maximum 
number of devices of course).

Regards,

Anthony Liguori

> It sounds less hacky to just extend the PCI slots instead of (ab)using
> multiple functions per-slot.
>   


-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference 
Don't miss this year's exciting event. There's still time to save $100. 
Use priority code J8TL2D2. 
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone

Re: [PATCH] Make virtio devices multi-function

[Avi Kivity]

Anthony Liguori wrote:
> BTW, I've never been that convinced that hotplugging devices is as 
> useful as people make it out to be.  I also think that's particularly 
> true when it comes to hot adding/removing very large numbers of disks.
>
>   

On the contrary, the more disks you have, the more likely one is to 
fail, so you'd need to hotreplace it (think a setup with redundancy like 
zfs).

-- 
Any sufficiently difficult bug is indistinguishable from a feature.


-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference 
Don't miss this year's exciting event. There's still time to save $100. 
Use priority code J8TL2D2. 
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone

Re: [PATCH] Make virtio devices multi-function

[Luca Tettamanti]

On Tue, Apr 22, 2008 at 4:15 PM, Anthony Liguori <aliguori@us.ibm.com> wrote:
> This patch changes virtio devices to be multi-function devices whenever
>  possible.  This increases the number of virtio devices we can support now by
>  a factor of 8.
[...]
>  diff --git a/qemu/hw/virtio.c b/qemu/hw/virtio.c
>  index 9100bb1..9ea14d3 100644
>  --- a/qemu/hw/virtio.c
>  +++ b/qemu/hw/virtio.c
>  @@ -405,9 +405,18 @@ VirtIODevice *virtio_init_pci(PCIBus *bus, const char *name,
>      PCIDevice *pci_dev;
>      uint8_t *config;
>      uint32_t size;
>  +    static int devfn = 7;
>  +
>  +    if ((devfn % 8) == 7)
>  +       devfn = -1;
>  +    else
>  +       devfn++;

This code look strange... devfn should be passed to virtio_init_pci by
virtio-{net,blk} init functions, no?

Luca

-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference 
Don't miss this year's exciting event. There's still time to save $100. 
Use priority code J8TL2D2. 
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone

Re: [PATCH] Make virtio devices multi-function

[Ryan Harper]

* Anthony Liguori <aliguori@us.ibm.com> [2008-04-22 09:16]:
> This patch changes virtio devices to be multi-function devices whenever
> possible.  This increases the number of virtio devices we can support now by
> a factor of 8.
> 
> With this patch, I've been able to launch a guest with either 220 disks or 220
> network adapters.

Have you confirmed that the network devices show up?  I was playing
around with some of the limits last night and while it is easy to get
QEMU to create the adapters, so far I've only had a guest see 29 pci
nics (e1000).


-- 
Ryan Harper
Software Engineer; Linux Technology Center
IBM Corp., Austin, Tx
(512) 838-9253   T/L: 678-9253
ryanh@us.ibm.com

-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference 
Don't miss this year's exciting event. There's still time to save $100. 
Use priority code J8TL2D2. 
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone

Re: [PATCH] Make virtio devices multi-function

[Anthony Liguori]

Ryan Harper wrote:
> * Anthony Liguori <aliguori@us.ibm.com> [2008-04-22 09:16]:
>   
>> This patch changes virtio devices to be multi-function devices whenever
>> possible.  This increases the number of virtio devices we can support now by
>> a factor of 8.
>>
>> With this patch, I've been able to launch a guest with either 220 disks or 220
>> network adapters.
>>     
>
> Have you confirmed that the network devices show up?  I was playing
> around with some of the limits last night and while it is easy to get
> QEMU to create the adapters, so far I've only had a guest see 29 pci
> nics (e1000).
>   

Yup, I had an eth219

Regards,

Anthony Liguori



-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference 
Don't miss this year's exciting event. There's still time to save $100. 
Use priority code J8TL2D2. 
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone